Open mbarbero opened 2 years ago
vince-fugnitto
commented
2 years ago @mbarbero thank you for the discussion!
we noticed that the requests POST /api/-/query (which are Theia specifics AFAIK), leads to 2.5+MB of gzipped json responses.
I don't believe the post endpoint for query is used any longer (https://github.com/eclipse-theia/theia/pull/10325).
msujew
commented
2 years ago I imagine that a lot of users still use older versions of Theia (#10325 was only included in 1.19). Ideally during the next few months, people update their Theia installations and start using the GET endpoint, which is caching everything, reducing the load drastically. However, I'm not sure whether using cached responses reduces the bandwidth-load considerably.
spoenemann
commented
2 years ago The amount of data returned by the GET endpoint is the same, so we could have a closer look what information we really need and add additional options to control what is included in the response.
amvanbaren
commented
2 years ago I can add offset and size query parameters to add paging to the /api/-/query endpoint. The /api/-/search endpoint has the same parameters to limit the response size. offset and totalSize are added to the query response, so that you can page through the extensions.
On the Theia side you then need to implement paging for the getAllVersions function.
https://github.com/eclipse-theia/theia/blob/bf00fd0fcf5b8153178b9bab05212da1ea9ceee8/dev-packages/ovsx-client/src/ovsx-client.ts#L89
The Eclipse Foundation is operating open-vsx.org and we noticed that the requests
POST /api/-/query(which are Theia specifics AFAIK), leads to 2.5+MB of gzipped json responses. This consume a large chunk of the Eclipse Foundation bandwidth to the detriment of other services / projects.Would it be possible to reduce the amount of data retrieved by those requests? Would it need some changes on the openvsx server to be more efficient?