kesara
commented
1 year ago The interval used to be even smaller I think 1 hour before. But the latest change to move to a different GHA made the action really slow. Sometimes it takes more than 3 hours to complete and that produces errors. I set it to 3 hours because it seems like a sweet spot with relatively low errors.
We need this GHA to run more frequently so authors can use references for new drafts.
What's required is to make this GHA faster so we can run this hourly.
This workflow takes between 3 and 5 hours to execute. In some cases (e.g. https://github.com/ietf-tools/relaton-data-ids/actions/runs/5539700257) a workflow run starts before the previous run is completed. This causes data inconsistency issues when the latter workflow runs.
It makes sense to either increase the interval between each workflow (e.g. this PR increases it to 6 hours), or schedule it to run periodically once a day, like it's done in the relaton org (https://github.com/relaton/relaton-data-ids/blob/52b1dd9bc2cb2c2c70d6dbfcf4893838ef6f00ae/.github/workflows/crawler.yml#L11).
@kesara any thoughts on this one? I see you recently changed the interval schedule to 3 hours, so maybe there are reasons I am not aware of for such a change.