microsoft / winget-cli

WinGet is the Windows Package Manager. This project includes a CLI (Command Line Interface), PowerShell modules, and a COM (Component Object Model) API (Application Programming Interface).
https://learn.microsoft.com/windows/package-manager/
MIT License
23.06k stars 1.43k forks source link

PreIndexed package schema modifications to reduce data transfer #4348

Closed denelon closed 2 months ago

denelon commented 6 months ago

Description of the new feature / enhancement

As the number of packages in the community repository increases, additional optimizations can be made to the structure of the index to reduce the data transfer needed to support local operations.

The default TTL for the community repository is 5 minutes. On average, the index is updated about every 17 minutes. Given this data, the default TTL should also be reduced to half that time.

Proposed technical implementation details

Newer versions of the client should be able to take advantage of the new schema with the smaller index. Earlier versions of WinGet will continue to use the existing schema and index. The publishing service should continue to publish both formats of the index. In the event this results in an extended publishing pipeline time, the index should be extended to match half the average publish pipeline time.

[!CAUTION] Applications shouldn't expect the schema for the enclosed SQLlite database to remain consistent from the point of automation. The schema is routinely optimized for client performance. Taking a dependency on the schema could result in unstable operation.

Trenly commented 6 months ago

Just one thought I had . . . It would probably be best if the TTL could be changed dynamically outside of the CLI. Either through a new entry in the metadata table in the index, or an HTTP header, or even just an API endpoint

denelon commented 3 months ago

The new v2 schema has been deployed, and we've tested with several clients. This will be the default in the preview builds of WinGet 1.9.

denelon commented 2 months ago

This feature has been completed and is in the latest WinGet 1.8 stable version as well as WinGet 1.9-preview.