openmeterio / openmeter

Cloud Metering for AI, Billing and FinOps. Collect and aggregate millions of usage events in real-time.
https://openmeter.io
Apache License 2.0
1.08k stars 60 forks source link

Add pagination to event queries #1527

Open Pascualex opened 2 days ago

Pascualex commented 2 days ago

Preflight Checklist

Problem Description

When retrieving a list of events trough the API you need to specify a limit for the number of elements returned. This can work fine for very constrained queries, but doesn't scale to high numbers of results.

Proposed Solution

Add pagination parameters to the query (e.g. page-size and page) to be able to retrieve specific parts of the query result.

Alternatives Considered

With the current API, it should be possible to implement infinite scrolling by querying for the events older than the last event retrieved. However, this seems hacky and doesn't support other types of pagination (like jumping directly to page n).

Additional Information

Although the existing /query endpoint is a great way to power visual dashboards, we believe that being able to properly navigate disaggregated usage records is equally important and currently lacking.

hekike commented 1 day ago

@Pascualex, thanks for opening the issue. We are planning to improve pagination for the events endpoint. Initially, this endpoint was meant to be used for debugging only and to show the last couple of events.

What is your use-case for getting more historical events?