CenterForOpenScience / SHARE

SHARE is building a free, open, data set about research and scholarly activities across their life cycle.
http://share-research.readthedocs.io/en/latest/index.html
Apache License 2.0
100 stars 58 forks source link

[SHARE-1082] Handle OSF harvest errors #766

Closed aaxelb closed 6 years ago

aaxelb commented 6 years ago

Sometimes OSF nodes are deleted or made private in the middle of a harvest, which breaks that day's harvest. This happens often, and is probably why we're missing so many OSF projects in SHARE.

When fetching embedded lists causes a 401 or 410 error, skip that node and casually continue the harvest.

https://openscience.atlassian.net/browse/SHARE-1082