repman-io / repman

Repman - PHP Repository Manager: packagist proxy and host for private packages
https://repman.io
MIT License
515 stars 107 forks source link

API rate limiting when importing a package from a gitlab.com repository #577

Open nstCactus opened 2 years ago

nstCactus commented 2 years ago

When importing a new package from a private gitlab.com repository, most of the time I get an error during the package syncrhonization step that immediately follows.

Error: Failed to download vendor/package from https://gitlab.com/api/v4/projects/path%2Fto%2Frepository/repository/archive.zip?sha=5e925b01ce8143e5e3caf36b86dc840be8c33559

When downloading this URL using my brwoser, everything works fine.

Digging in the output of the bin/console messenger:consume -vvv command output, I found the following error:

11:01:13 DEBUG [php] Warning: fopen(https://gitlab.com/api/v4/projects/path%2Fto%2Frepository/repository/archive.zip?sha=5e925b01ce8143e5e3caf36b86dc840be8c33559): failed to open stream: HTTP request failed! HTTP/1.1 429 Too Many Requests

After waiting for some time and forcing a new sync, it usually succeeds, but it's a pain having to go through this every time.

I am the only one facing this issue? Is there any setting I can change in either gitlab.com or repoman to mitigate this issue?

hugodeaguiar commented 2 years ago

Hi @nstCactus!

I have same issue here!

The solution for this problem will be add a little interval between requests to Gitlab. But I didn't identify the file that makes this request yet;

stigfaerch commented 2 years ago

Same here.

nstCactus commented 1 year ago

I found a possible workaround: getting only the last ten releases. This is less than ideal…