fog / fog-google

Fog for Google Cloud Platform
MIT License
99 stars 146 forks source link

Unable to list more than 500 instances with `Fog::Compute::Google::Servers#all` #607

Closed agrare closed 8 months ago

agrare commented 9 months ago

If there are more results than will fit in a page (default size 500 but can be limited with :max_results) then a nextPageToken is returned with the InstanceList.

The Fog::Compute::Google::Servers#all method ignores this and returns the items in the first page only.

There is an optional :page_token parameter to #all but the next_page_token isn't returned to the caller so it isn't obvious how this was intended to be used.

I don't personally have an environment with this many instances but the issue can be easily demonstrated by using :max_results to force list_servers and list_aggregated_servers to return a smaller page with a nextPageToken:

>> compute.servers.all.count
=> 4
>> compute.servers.all(:max_results => 1).count
=> 1

In this case max_results limits service.list_aggregated_servers to return a single server plus a nextPageToken which is ignored. If the page size were not arbitrarily limited then only the first 500 instances would be returned.

This is fundamentally the same issue as https://github.com/fog/fog-google/issues/221 and https://github.com/fog/fog-google/issues/42

I believe all of the Model#all methods likely have this same issue.