ucldc / rikolti

calisphere harvester 2.0
BSD 3-Clause "New" or "Revised" License
7 stars 3 forks source link

ETag checks: allow redirects and add Nuxeo workaround #1140

Closed barbarahui closed 1 month ago

barbarahui commented 1 month ago

By default requests will perform location redirection for all verbs except HEAD, so I've added allow_redirects=True to the etag check request.

Also, it turns out that S3 presigned URLs for GET requests are different from those for HEAD requests. The Nuxeo API redirects us to a presigned S3 URL, which returns a 403 because it's only good for GETs. The easiest thing I could figure out to get around this was to issue a GET request that only returns the headers (basically mimicking the behavior of a HEAD request).