search

wbond / package_control

The Sublime Text package manager
https://packagecontrol.io
4.77k stars 816 forks source link

A github repo slows "update all packages" #1638

Open reagle opened 1 year ago

reagle commented 1 year ago

Following the directions to install a dev plugin from github results in very slow package control.

In STealthy-and-haSTy/EnhancedSnippets/issues/7 @OdatNurd suggests its a PC issue.

macOS 13.4 22F66 arm64

FichteFoll commented 1 year ago

It's likely that you end up being rate-limited and there's nothing that PC can do about it by itself. However, you could provide PC with some credentials for GitHub's API in your preferences.

https://github.com/wbond/package_control/blob/cfaaeb57612023e3679ecb7f8cd7ceac9f57990d/Package%20Control.sublime-settings#L91-L103

deathaxe commented 1 year ago

_FWIW: IIRC, basic auth doesn't work properly in PC 3.x (see: https://github.com/wbond/package_control/pull/1622)._

That said, resolving release information from VCS based releases requires various API calls per package, which are performed one after another. Even if rate-limit is not hit, it just takes some time.

PC4.0 takes some meassures to reduce amount of API calls and runs some stuff in parallel to reduce delays, but finally it would require a complete rewrite applying asyncio or apply other complicated Promise architectures to solve the problem.

I don't think any of that will happen anytime soon, especially as PC4.0 claims support for ST3143+ and needs to stay fully compatible with python 3.3 for the forseeable time. The other reason is likelyness of hitting rate limits. Even authenticated users can only perform 5000 API calls per hour, which is not enough to resolve the whole default channel's repositories atm.

Such kind of performance issues is the main reason for existence of channels as those contain a cache for resolved packages. Unfortunatelly packagecontrol.io is currently the only channel provider, which implements the background crawler to fill those caches.

The only other solution would be to provide explicit releases via repository.json, but this would also require to either hand craft repositories after each release or create a deployment script doing so.

An example for such repositories is https://github.com/SublimeText/wbond-packages/blob/main/packages.json

reagle commented 1 year ago

This morning I'm seeing this problem, which might be related?

Package Control: Hit rate limit of 60 for api.github.com. Skipping all futher download requests for this domain.
deathaxe commented 1 year ago

This indicates you not being able to even see all packages from that repo.

There's nothing Package Control can do against hitting a repository hoster's API rate limits.

The only issue is basic auth not working for PC3.x Thus being restricted to lowest available rate limits. This issue is fixed in PC4.0