search

harvard-lil / perma

Indelible links
420 stars 71 forks source link

Upload files to and delete files from "daily" digest IA items, Part X #3289

Closed rebeccacremona closed 1 year ago

rebeccacremona commented 1 year ago

This PR fixes an oversight in this series: the queuing function assumes that at least one link will always be queued. If it isn't, this logging line throws IndexErrors.

Instead of just fixing the log line, I decided to add special handling for that case. When no eligible links are found for a given day, we provisionally mark that day's InternetArchiveItem "complete."

That metadatum won't necessarily be true, going forward: planned future work, which will handle Perma Links that are toggled from public to private and vice versa, will be toggling this field and triggering the appropriate work... But, it is useful in the time being: we can use it to determine where in the backlog to start queuing, when conditionally_queue_internet_archive_uploads_for_date_range runs.

This PR arranges for conditionally_queue_internet_archive_uploads_for_date_range to look for the most recent 'complete' day, and start the day after that.

Deploying

Before this is deployed, I will go into the Django admin or the shell, and mark the 4 days we finished working on last night "complete".

rebeccacremona commented 1 year ago

I might need to move the code about timing the query; checking. Actually, this might cause the query to be evaluated twice. Checking that as well.

rebeccacremona commented 1 year ago

Yup, I need to do both of those things. Hold one.

codecov[bot] commented 1 year ago

Codecov Report

Merging #3289 (e2d598b) into develop (e88bd06) will decrease coverage by 0.17%. The diff coverage is 2.85%.

@@             Coverage Diff             @@
##           develop    #3289      +/-   ##
===========================================
- Coverage    74.11%   73.94%   -0.17%     
===========================================
  Files           55       55              
  Lines         6591     6606      +15     
===========================================
  Hits          4885     4885              
- Misses        1706     1721      +15
Impacted Files Coverage Δ
perma_web/perma/celery_tasks.py 54.31% <2.85%> (-0.69%) :arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

rebeccacremona commented 1 year ago

Okay, addressed. Still LGTY, @bensteinberg?