search

iftechfoundation / ifarchive-ifmap-py

The tool that generates the index files at http://ifarchive.org/ .
3 stars 1 forks source link

uncache.py: Too many files in request #35

Closed erkyrath closed 7 months ago

erkyrath commented 7 months ago

uncache -z foo.zip tries to uncache https://unbox.ifarchive.org/HASH/FILE for every file inside foo.zip.

If this is too many files, we get an error:

HTTP Error 400: Bad Request {"success":false,"errors":[{"code":1094,"message":"Exceeded maximum amount of files that can be purged on a single request for your plan type."}],"messages":[],"result":null}

I haven't looked up the limit, but we hit it with 56 files in the request: https://ifarchive.org/if-archive/games/zcode/spanish/Protector.zip

Gotta batch that.

(Also, we should catch and display exceptions from urllib.request.urlopen().)

erkyrath commented 7 months ago

Batching at 16 files per request.

We display errors when running from the command line. The admin tool still drops them silently -- sorry.