search

nats-io / nats.ex

Elixir client for NATS, the cloud native messaging system. https://nats.io
MIT License
202 stars 36 forks source link

Support pagination for KV and Object `list_bucket` #172

Open jcreager opened 2 weeks ago

jcreager commented 2 weeks ago

Proposal:

list_buckets for both KV and Object have their own contract that looks like {:ok, ["TEST_BUCKET_1", "TEST_BUCKET_2"], %{total:_, offset: 0, limit: 1024}}.

list_buckets/2 in either module will optionally accept a parameter which specifies the offset and limit of the next page.

This is to avoid introducing breaking changes to the existing implementation of KV's list_object function, as well as the proposed Object list_buckets function.

mmmries commented 2 weeks ago

I was thinking about this a bit more and one way we could keep the API the same is to handle pagination inside the function itself. That seems to be what this function in go does: https://github.com/nats-io/nats.go/blob/f0b1ea421c52eee17f89d528543b7affba18910b/kv.go#L1150-L1167

It paginates through all the streams and for each of them that match the right prefix, it generates a KeyValueStoreStatus. We're just returning the names, but I think we could handle the pagination and then just return the full list of KV bucket names?

I also couldn't find any pagination options that are available on the CLI, so I'm guessing it just paginates through all of them as well?