At the moment when there's a change in the schedule, each of the 600+ calendar files will be updated, even if the old file and the new file are identical. It's quite common for there to be no change to the old and new files because the Cape Town loadshedding schedules and the Eskom loadshedding schedules are largely disjoint, so if Eskom announces an update then none of the CPT schedules will be changed.
This is a problem because uploading the files is the most expensive step of the release process. Uploading takes up to 10minutes each time, and it also consumes the hourly rate limit. In the past, there have been times when Eskom and Cape Town announced multiple schedule updates within an hour, and eskom-calendar exceeded the GitHub rate limit quota, and so had to wait about 30 minutes until the quota got reset and the files could be uploaded. Also, the number of calendar files isn't going down any time soon, so this problem of the rate limit quota will only get worse.
This could be fixed by modifying the beyarkay/update-existing-release repo to also upload a cachefile .cache which would be a YAML-formatted file containing a list of all the files, their hashes, and the last time they were updated. Logic would need to be added to only upload a file if the new hash and the existing hash have changed.
Ideally this change would be upstreamed to the original IsaacShelton/update-existing-release repo, but the in-house fork beyarkay/update-existing-release is quite behind and I'm not sure if upstreaming these changes might cause other issues.
At the moment when there's a change in the schedule, each of the 600+ calendar files will be updated, even if the old file and the new file are identical. It's quite common for there to be no change to the old and new files because the Cape Town loadshedding schedules and the Eskom loadshedding schedules are largely disjoint, so if Eskom announces an update then none of the CPT schedules will be changed.
This is a problem because uploading the files is the most expensive step of the release process. Uploading takes up to 10minutes each time, and it also consumes the hourly rate limit. In the past, there have been times when Eskom and Cape Town announced multiple schedule updates within an hour, and eskom-calendar exceeded the GitHub rate limit quota, and so had to wait about 30 minutes until the quota got reset and the files could be uploaded. Also, the number of calendar files isn't going down any time soon, so this problem of the rate limit quota will only get worse.
This could be fixed by modifying the
beyarkay/update-existing-releaserepo to also upload a cachefile.cachewhich would be a YAML-formatted file containing a list of all the files, their hashes, and the last time they were updated. Logic would need to be added to only upload a file if the new hash and the existing hash have changed.Ideally this change would be upstreamed to the original
IsaacShelton/update-existing-releaserepo, but the in-house forkbeyarkay/update-existing-releaseis quite behind and I'm not sure if upstreaming these changes might cause other issues.