EclipseFdn / open-vsx.org

Source of open-vsx.org
https://open-vsx.org/
Eclipse Public License 2.0
139 stars 40 forks source link

SLA of open-vsx.org #2859

Closed cdietrich closed 1 month ago

cdietrich commented 2 months ago

We are running builds on github actions that produce docker images with code-server and some preinstalled extension in our dockerfile we do something like

RUN ["code-server", "--install-extension", "dbaeumer.vscode-eslint@2.4.4"]
RUN ["code-server", "--install-extension", "esbenp.prettier-vscode"]
RUN ["code-server", "--install-extension", "vscode-icons-team.vscode-icons"]
RUN ["code-server", "--install-extension", "humao.rest-client"]
RUN ["code-server", "--install-extension", "ms-python.python"]

this sometime fails with "Too many request".

[release 19/48] RUN ["code-server", "--install-extension", "ms-python.python"]:
0.400 Installing extensions...
0.706 Extension 'ms-python.python' not found.
0.706 Make sure you use the full extension ID, including the publisher, e.g.: ms-dotnettools.csharp
0.708 Failed Installing Extensions: ms-python.python

what are the current SLAs and is there something we can do to avoid the problem.

initially reported as https://github.com/eclipse/openvsx/issues/975

kineticsquid commented 2 months ago

@cdietrich Sorry for the delay in getting back to you on this. Do the extensions that fail differ run to run and is it always an extension not found error? And you're attempting to install from open-vsx.org directly?

@amvanbaren Thoughts?

amvanbaren commented 2 months ago

is there something we can do to avoid the problem

@cdietrich Yes, you can use the X-Rate-Limit-Remaining and X-Rate-Limit-Retry-After-Seconds response headers to prevent the 'Too Many Requests' status, as described here: https://open-vsx.org/swagger-ui/index.html#/registry-api/getFile

cdietrich commented 2 months ago

@amvanbaren so should this be implemented inside vscode/codiums/code-server install-extension commands?

amvanbaren commented 2 months ago

@cdietrich I would first try to find out how you're able to install 15 extensions per second.

cdietrich commented 2 months ago

if this runs in github actions. then there might be a lot of different actions from different github accounts try to do the same thing, using the same ip, wont they?

we will try to throttle on our side anyway and keep observing

amvanbaren commented 2 months ago

Github has about 3800 IP ranges for actions: https://api.github.com/meta. While it's not infinite, it's still a lot. It could be that an organization is assigned a specific IP address for its actions. You can find out by adding an extra step to the workflow: https://swharden.com/blog/2022-03-12-github-actions-ip/

kineticsquid commented 1 month ago

@cdietrich OK to close this now?

kineticsquid commented 1 month ago

@cdietrich ^^^^

cdietrich commented 1 month ago

It’s ok to close

kineticsquid commented 1 month ago

Thanks!