search

regosen / get_cover_art

Batch cover art downloader and embedder for audio files
MIT License
69 stars 8 forks source link

Getting 403 forbidden from itunes.apple.com, requesting too fast? #16

Closed codeswhite closed 2 years ago

codeswhite commented 2 years ago

I just might implement proxy usage...

vext01 commented 2 years ago

I get this too when running on a large collection.

regosen commented 2 years ago

Have you tried the --throttle option? It specifies the number of seconds to wait between each query (i.e. each album). It defaults to 0, but if you find a good number that works, I could update the default value.

regosen commented 2 years ago

I’m also considering responding to a 403 by sleeping for a long time before trying again (and informing the user of course). Does Apple indicate how long it locks you out for?

On Nov 29, 2021, at 10:13 AM, Edd Barrett @.***> wrote:

 I get this too when running on a large collection.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

regosen commented 2 years ago

Ok, I just pushed a new version (1.5.0) that will automatically sleep for 30 minutes and retry whenever it encounters a 403 or 429.

From what I'm reading, Apple enforces a limit per hour, but I'm not sure if it's on absolute hour intervals or relative to the first request, so 30 minutes should be a good duration for this. (It will sleep again if the retry hits another 403 or 429)

codeswhite commented 2 years ago

I found out that the 403 is returned from the server if I have mady more than X request in the last Y seconds. It's enough to wait just few seconds after 403.

A good throttle value i found to be around 1-2 seconds, maybe a little more on faster computers.

Anyway I have forked your code, refactored it and rebranded it to my needs... I've stripped local cover finding and prioritizing as I don't have those, only embedded. Added multiple download sources besides Apple, image optimization (size minification) after download, about 20 command line options, integration with youtube_dl, multithreading and more..

I will probably publish it on my profile, maybe we can merge projects

vext01 commented 2 years ago

Might be worth making the wait time configurable and lowering the default: seems you can re-request much quicker than 30 minutes when you get a 403 (10 seconds?).

regosen commented 2 years ago

Good call, I just reintroduced --throttle which will default to 3 but not kick in until a 403 or 429 are encountered.

@codeswhite good call on stripping out all the "local cover finding" options, that made the instructions really complicated and I'm not sure how useful they are. (They were introduced by another user, so I know one person used them at least.)