[SHARE-1007][Fix] Add x-throttle-token header to OSF harvester

CenterForOpenScience / SHARE

SHARE is building a free, open, data set about research and scholarly activities across their life cycle.

http://share-research.readthedocs.io/en/latest/index.html

Apache License 2.0

100 stars 58 forks source link

[SHARE-1007][Fix] Add x-throttle-token header to OSF harvester #734

Closed laurenbarker closed 6 years ago

laurenbarker commented 6 years ago

Requires that devops adds the BYPASS_THROTTLE_TOKEN as an environment variable on prod. Michael has been notified. ✅ this has been done

Purpose

OSF harvest tasks fail, every day. We are getting 429s from the OSF API.

Changes

Add no throttle header

QA notes

Doesn't need QA.

coveralls commented 6 years ago

Coverage decreased (-0.01%) to 79.02% when pulling 1e8789cac2b40c771f6e1d186bc6673096fa12d6 on laurenbarker:fix/SHARE-1007 into 3110b5bc29448ff9100dbfc06fc27b9b2a1d66ef on CenterForOpenScience:future-release/2.14.0.

laurenbarker commented 6 years ago

@icereval @binoculars this is the throttle token PR scheduled for v2.14.0 (does not currently have a release date). We will need the throttle token added to the production environment. We don't get throttled on staging so no need to add it there.

Anything we should be worried about? We have the rate limit for OSF set to 1 request every 3 seconds.

coveralls commented 6 years ago

Coverage decreased (-0.01%) to 79.02% when pulling 0156a242cc3c931db22c20eb618bc4a6c009a98e on laurenbarker:fix/SHARE-1007 into 3110b5bc29448ff9100dbfc06fc27b9b2a1d66ef on CenterForOpenScience:future-release/2.14.0.

coveralls commented 6 years ago

Coverage decreased (-0.01%) to 78.892% when pulling 74394aa281f5744220d086341745757e88d6148c on laurenbarker:fix/SHARE-1007 into 751916c0fc2cd24a9fb0a819379c3d3f5f3682db on CenterForOpenScience:develop.

coveralls commented 6 years ago