harvard-lil / perma

Indelible links
423 stars 71 forks source link

Remove unnecessary availability check #3655

Closed rebeccacremona closed 1 week ago

rebeccacremona commented 1 week ago

Many years ago, we were occasionally finding that, when we uploaded WARCs to S3, they were not immediately available for download, even though the upload was completed. We added a retry to account for this.

Also many years ago, but more recently, Jack looked in to it and reported S3 has since changed, and this inconsistency isn't a problem any more. We decided to keep this check in the codebase anyway, because it was harmless.

Well, now it's causing trouble: we don't have WACZs when new upload-your-own archives are created, because we are still just making WARCs for those. So, the check fails.

Instead of tweaking the check to work properly in both situations, I'm just removing it.

If we find that S3 is, in fact, still inconsistent, then we can add a check back in.

codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 69.61%. Comparing base (d4747b8) to head (e631bd3). Report is 5 commits behind head on develop.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #3655 +/- ## =========================================== - Coverage 69.61% 69.61% -0.01% =========================================== Files 54 54 Lines 7350 7339 -11 =========================================== - Hits 5117 5109 -8 + Misses 2233 2230 -3 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.