CrowCamCleanup has stopped working

tfabris / CrowCam

A set of Bash scripts to control and maintain a YouTube live cam from a Synology NAS.

GNU General Public License v3.0

3 stars 3 forks source link

CrowCamCleanup has stopped working #51

Closed tfabris closed 4 years ago

tfabris commented 4 years ago

I notice that old videos are not getting cleaned out by CrowCamCleanup any more. Every night I get the "50 videos found, processing..." message in the log, but no videos are being deleted in the process.

Need to debug.

tfabris commented 4 years ago

Aha, the issue is a "perfect storm" combination of two things:

The 50-item-per-page limit.
The fact that Comcast Cable has been flaky lately and there are a lot of video restarts each day.

This causes there to be more than 50 recent videos which are outside of the "deletion range". So when it scans the most recent 50 videos, they will always be recent videos and won't be deleted.

Original code design thought 50 would be enough, but it's not.

To do:

Figure out a way to get the code to query more than 50 videos back.
Might have to implement actual pagination.

tfabris commented 4 years ago

Checked in the pagination code fix. Discovered that it will outrun the Google quota if you do too many pages , so limit the number of pages.

This has been bench tested, and also used for deleting a large hunk of old pages left over from the bug happening for a while.

To do:

Get the new version copied into actual production server so that it runs every night.
In production, change the nightly runtime from midnight to 11pm, to take advantage of the Google midnight quota reset.

tfabris commented 4 years ago

Fixed! Works well when set to 3 pages (150 videos). At first, it was overrunning (or nearly overrunning) the quota due to having so many videos to clean up, but after the big bulk of the cleanup was done, the quota usage dribbled back down to 30 percent and now it's back under control. Bug fixed! Closing.