Most Variational API endpoints that can return multiple results limit the number of objects returned by a single call. This means that a pagination mechanism must be used to retrieve more objects.
Endpoints using pagination can be identified by the presence of the top-level pagination key in the JSON data they return. In the their Python SDK return value is marked as ApiPage[T].
Different endpoints return different number of items on a single page. The specifies this number for each endpoint.
Example
When requesting the first page of trades, the response will contain query string parameters required to request the next page.
Example first request:
GET https://api.testnet.variational.io/v1/portfolio/trades?pool=daf1b8ef-bac8-47a0-8d41-f18c9aed8442
For flexibility, the Python SDK Client methods don't perform the pagination automatically:
from variational import Client, TESTNET
client = Client(API_KEY, API_SECRET, base_url=TESTNET)
trades_resp = client.get_portfolio_trades(pool="daf1b8ef-bac8-47a0-8d41-f18c9aed8442")
trades = trades_resp.result # only the first page of trades
Manual Pagination
If necessary, the next page can be retrieved afterward:
To avoid boilerplate, the SDK provides a helper generator function paginate() that yields the requested objects making API calls lazily while it's being consumed.
This code fetches all trades from the previous example into a list using the helper function:
from variational import paginate
# Note that we don't call `client.get_portfolio_trades` here,
# but rather pass a reference to it to `paginate()`.
# All parameters meant for `client.get_portfolio_trades`
# should be passed to `paginate()`.
# `list()` is meant to consume all results from the generator
# at once and eagerly request all pages.
all_trades = list(paginate(client.get_portfolio_trades,
pool="daf1b8ef-bac8-47a0-8d41-f18c9aed8442"))
An example making use of lazy pagination:
pool_id = "daf1b8ef-bac8-47a0-8d41-f18c9aed8442"
for trade in paginate(client.get_portfolio_trades, pool=pool_id):
if is_this_the_last_trade_we_need(trade):
break # no more API calls requesting subsequent pages will happen after this
Logging
It might be useful to enable logging in your application to get an indication of progress.
A simple logging configuration that enables debug-level logging for all facilities:
import logging
# run this before initializing variational.Client
logging.basicConfig(
format="[%(levelname)s] %(asctime)s %(name)s %(message)s",
level=logging.DEBUG
)
Advanced configuration enabling only relevant loggers:
If you have a large amount of activity in your account, it might take multiple requests and a significant amount of time to fetch all resulting objects using pagination. Running multiple pagination requests back to back makes it likely to run into . The SDK Client retries them automatically by default but the entire operation will become slowed down.
