Note: This is a part of our series on API Pagination where we solve common developer queries in detail with common examples and code snippets. Please read the full guide here where we discuss page size, error handling, pagination stability, caching strategies and more.
There are several common API pagination techniques that developers employ to implement efficient data retrieval. Here are a few useful ones you must know:
1. Offset and Limit Pagination
This technique involves using two parameters: "offset" and "limit." The "offset" parameter determines the starting point or position in the dataset, while the "limit" parameter specifies the maximum number of records to include on each page.
For example, an API request could include parameters like "offset=0" and "limit=10" to retrieve the first 10 records.
GET /aCpi/posts?offset=0&limit=10
2. Cursor-Based Pagination
Instead of relying on numeric offsets, cursor-based pagination uses a unique identifier or token to mark the position in the dataset. The API consumer includes the cursor value in subsequent requests to fetch the next page of data.
This approach ensures stability when new data is added or existing data is modified. The cursor can be based on various criteria, such as a timestamp, a primary key, or an encoded representation of the record.
For example - GET /api/posts?cursor=eyJpZCI6MX0
In the above API request, the cursor value `eyJpZCI6MX0` represents the identifier of the last fetched record. This request retrieves the next page of posts after that specific cursor.
3. Page-Based Pagination
Page-based pagination involves using a "page" parameter to specify the desired page number. The API consumer requests a specific page of data, and the API responds with the corresponding page, typically along with metadata such as the total number of pages or total record count.
This technique simplifies navigation and is often combined with other parameters like "limit" to determine the number of records per page.
For example - GET /api/posts?page=2&limit=20
In this API request, we are requesting the second page, where each page contains 20 posts.
4. Time-Based Pagination
In scenarios where data has a temporal aspect, time-based pagination can be useful. It involves using time-related parameters, such as "start_time" and "end_time", to specify a time range for retrieving data.
This technique enables fetching data in chronological or reverse-chronological order, allowing for efficient retrieval of recent or historical data.
For example - GET/api/events?start_time=2023-01-01T00:00:00Z&end_time=2023-01-31T23:59:59Z
Here, this request fetches events that occurred between January 1, 2023, and January 31, 2023, based on their timestamp.
5. Keyset Pagination
Keyset pagination relies on sorting and using a unique attribute or key in the dataset to determine the starting point for retrieving the next page.
For example, if the data is sorted by a timestamp or an identifier, the API consumer includes the last seen timestamp or identifier as a parameter to fetch the next set of records. This technique ensures efficient retrieval of subsequent pages without duplication or missing records.
To further simplify this, consider an API request GET /api/products?last_key=XYZ123. Here, XYZ123 represents the last seen key or identifier. The request retrieves the next set of products after the one with the key XYZ123.
Also read: 7 ways to handle common errors and invalid requests in API pagination
FAQs
What is API pagination?
API pagination is a technique for splitting large datasets into smaller, sequential chunks (pages) so clients can retrieve them incrementally rather than fetching everything at once. Without pagination, a single API request on a large dataset can time out, exhaust memory, or return millions of records the client doesn't need. Pagination controls - like page numbers, offsets, or cursors - let the client request exactly the range of data it needs, keeping response times fast and server load manageable.
What are pagination techniques?
The main API pagination techniques are: offset and limit (skip N records, return the next M), page-based (request page 3 of 10), cursor-based (use an opaque pointer to the last-seen record), time-based (fetch records created/updated after a given timestamp), and keyset/seek pagination (filter by the value of a sortable indexed column). Each suits different use cases - cursor-based is best for real-time feeds and large datasets, offset works for simple sorted results, and time-based is ideal for incremental data sync.
What are the different types of pagination?
The five most common types are:
(1) Offset pagination - uses offset and limit parameters, simple to implement but degrades on large datasets due to full table scans;
(2) Page-based pagination - uses page and per_page, conceptually simple but has the same performance limitations as offset;
(3) Cursor-based pagination - uses an opaque cursor token pointing to the last record, stable and performant even on large or frequently-updated datasets;
(4) Time-based pagination - fetches records within a time window using since and until parameters;
(5) Keyset pagination - filters by the value of an indexed column, combining the stability of cursors with direct SQL efficiency.
How to do pagination on API?
To implement pagination on an API: choose a pagination style (offset, cursor, or keyset depending on your dataset size and update frequency), add the relevant query parameters to your GET endpoint (e.g. ?limit=100&offset=0 or ?after=cursor_token), return pagination metadata in the response (total count, next cursor or next page URL), and handle the last page by returning an empty next cursor or a has_more: false flag. On the client side, follow the next link or cursor in each response until no further pages are returned.
What are the advantages of cursor-based pagination over offset pagination?
Cursor-based pagination has three key advantages over offset:
- Stability - if records are inserted or deleted between page requests, offset pagination skips or duplicates records; cursors point to a specific position so page boundaries remain consistent;
- Performance - offset pagination requires the database to scan and discard all preceding rows, which is slow on large tables; cursor-based queries use indexed lookups;
- Consistency at scale - cursor pagination works reliably on datasets with millions of records where offset becomes prohibitively slow.
The tradeoff is that cursor pagination doesn't support random page access or total record counts as easily.
What are API pagination best practices?
Key best practices: use cursor-based or keyset pagination for large or frequently-updated datasets rather than offset; always return a next cursor or link in the response so clients don't need to calculate the next page themselves; set a sensible default and maximum page size (e.g. default 100, max 1000) to prevent unbounded requests; include a has_more boolean or empty next to signal the final page clearly; use consistent parameter names (limit, after, before) so clients don't need to re-learn the interface per endpoint; and document the pagination model explicitly, since different endpoints on the same API sometimes use different styles.
When should I use time-based pagination?
Time-based pagination is best suited for incremental data sync use cases - where you want to fetch only records created or updated after a specific timestamp, rather than fetching all records from scratch on each run. It's commonly used in webhook alternative patterns, audit log retrieval, and integration sync loops. The main limitation is that it assumes records have reliable, indexed created_at or updated_at timestamps, and it can miss records if clock skew or delayed writes cause them to land before the since boundary.
How does API pagination affect integration performance?
Pagination style significantly affects integration performance. Offset pagination becomes slow on large tables and can produce inconsistent results under concurrent writes - a common problem when syncing employee data from HRIS platforms that update frequently. Cursor-based pagination is more reliable for integration sync loops because it handles insertions and deletions between pages gracefully. When building integrations against third-party APIs, always check which pagination style they use and implement retry logic with backoff for rate-limited page requests. Knit manages all kinds of pagination for you when you're running syncs on Knit so you don't have to worry about how different apps might behave.
.webp)
