Read all pages of a package's results

[bgrainger]

Fixes #95.

When the NuGet API returns package versions as part of the response, the code only read the versions from the last page. It now reads the versions from all pages, whether they're returned inline or referenced via a secondary URL.

[praeclarum]

Cool!

So truth be told, I did that because I was worried about performance. But I think that’s when I didn’t have lazy loading of packages. I’m assuming performance is still good with this? Are there any packages in particular I should test out?

[bgrainger]

Good question; I didn't test that.

serilog (https://www.fuget.org/packages/serilog/) has 341 package versions. (I think, but don't know for sure, that this is on the high end for typical NuGet packages.) Testing locally, this code loaded six pages of results in ~1.45s, so about 250ms per page. The HTTP requests to the NuGet API were about 65ms each. If the rest of the time was parsing and loading the results, there may be scope for optimisation in FuGetGallery. (I haven't profiled it in detail.)

[loic-sharma]

Some more packages with 1000+ versions:

• https://www.nuget.org/packages/FAKE/
• https://www.nuget.org/packages/Igor.Gateway.Dtos/
• https://www.nuget.org/packages/Habanero/

[praeclarum]

Given what @loic-sharma said, maybe we should limit this to a couple pages. I don't see the utility of a UI presenting 1000 versions.

[loic-sharma]

Just curious, why are you using the package metadata (registration) API to get the list of versions? FYI, you can also get the list of versions using the package content (flat container) API or the autocomplete API:

• Autocomplete - enumerates all LISTED versions
  • This is not supported by all package sources (like Sleet)
  • Docs: https://docs.microsoft.com/en-us/nuget/api/search-autocomplete-service-resource#enumerate-package-versions
  • Example URL: https://azuresearch-usnc.nuget.org/autocomplete?id=mysql.data
  • Example client: https://github.com/loic-sharma/BaGet/blob/9c5a9f38896a5b70f6bc66b17c1c94543af39c2a/src/BaGet.Protocol/Search/RawAutocompleteClient.cs#L67
• Package content - enumerates ALL versions
  • This works on all V3 package sources
  • Docs: https://docs.microsoft.com/en-us/nuget/api/package-base-address-resource#enumerate-package-versions
  • Example URL: https://api.nuget.org/v3-flatcontainer/mysql.data/index.json
  • Example client: https://github.com/loic-sharma/BaGet/blob/9c5a9f38896a5b70f6bc66b17c1c94543af39c2a/src/BaGet.Protocol/PackageContent/RawPackageContentClient.cs#L33

[praeclarum]

I don't know any specific reason other than ignorance. In my defense, "Autocomplete" is not an API name I'd associate with version listings. ;-)

Thanks for the tips! I'll add this when switching over to service provider lookups.

[loic-sharma]

In my defense, "Autocomplete" is not an API name I'd associate with version listings. ;-)

Autocomplete can do both autocomplete on package IDs and enumerate listed versions. I agree it's really confusing 😂

[praeclarum]

@loic-sharma I'm not able to use either of the APIs you mentioned because neither provides date information for the versions (I display their publish date in the UI).