Part II worked: we were able to successfully confirm that the first batch of 10 files was uploaded, and were able to successfully upload and confirm a second batch of 10.

But, in the short period of time that ran, I saw almost 500 confirm tasks transiently accumulate in the queue. That shouldn't happen: only one confirm task should be scheduled per upload, and it should either successfully run or reschedule itself.

I think it's all due to the missing return added in this PR: tasks were rescheduling themselves on failure but then continuing. I'm a little surprised that would be enough to get us all the way up to 500... but I don't see any other problems. Let's try another batch of 10 with this fix in place and see if it happens again.

For redundancy, I also added a check at the beginning of the task: if this file has already been confirmed present, quit, rather than redundantly completing the rest of the task.

Codecov Report

Merging #3242 (468468b) into develop (176d1fd) will decrease coverage by 0.03%. The diff coverage is 0.00%.

:exclamation: Current head 468468b differs from pull request most recent head ee000e8. Consider uploading reports for the commit ee000e8 to get more accurate results

@@             Coverage Diff             @@
##           develop    #3242      +/-   ##
===========================================
- Coverage    79.21%   79.17%   -0.04%     
===========================================
  Files           53       53              
  Lines         6144     6147       +3     
===========================================
  Hits          4867     4867              
- Misses        1277     1280       +3

Impacted Files	Coverage Δ
perma_web/perma/tasks.py	`53.11% <0.00%> (-0.13%)`	:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

harvard-lil / perma

Upload files to existing "daily" digest IA items, Part III #3242

Codecov Report