Continue trying to diagnose & fix timeout problem

[jepler]

The latest failed result had [long lines are truncated]

2023-11-29T10:04:45.5181350Z GET /repos/adafruit/Adafruit_CircuitPython_ESP32SPI
2023-11-29T10:04:45.5182341Z 1 requests remaining this hour
2023-11-29T10:04:45.5183097Z GitHub API Rate Limit reached. Pausing until Rate L
2023-11-29T10:04:45.5183863Z Rate Limit will reset at: 2023-11-29 10:21:42
2023-11-29T10:04:45.5184590Z Sleeping 1016 seconds
2023-11-29T10:21:42.6205338Z Resetting dropped connection: api.github.com
2023-11-29T10:21:42.8508270Z https://api.github.com:443 "GET /repos/adafruit/Ada
2023-11-29T10:21:42.8564161Z GET https://api.github.com/repos/adafruit/Adafruit_
2023-11-29T10:21:42.8953319Z https://api.github.com:443 "GET /repos/adafruit/Ada
2023-11-29T10:21:42.8970216Z GET https://api.github.com/repos/adafruit/Adafruit_
2023-11-29T10:21:42.9467791Z https://api.github.com:443 "GET /rate_limit HTTP/1.
2023-11-29T10:21:42.9481187Z GET https://api.github.com/rate_limit {'Authorizati
2023-11-29T10:21:42.9490403Z Rate Limit will reset at: 2023-11-29 10:21:42
2023-11-29T15:21:15.6703237Z ##[error]The operation was canceled.

the first message is from our own lib github_requests.py because it says "1 requests remaining this hour". However, apparently it didn't wait long enough, because the very next request was still rate-limited. This request was via GH_INTERFACE, and the sleep was SUPPOSED to be corrected by my previous change to do arithmetic between the core.reset datetime and a datetime.utcnow timestamp.

Multi-pronged approach:

• Add 60s to all rate limit sleeps to avoid double rate limiting (allows clock skew between github actions server & github API server)
• limit all rate limit sleeps to no more than 3600 seconds to avoid multi-hour hangs (corrects for bad time arithmetic)
• print the rate limit time, the "now" time, and the "utcnow" time to diagnose what times and timestamps are involved

[jepler]

looks like it's possible to test this by updating adabot to this branch in circuitpython-org, then requesting to run the update-bundles workflow from that branch. so at least we don't have to test merge every time we're trying something different to fix it (as long as we don't care about ~8000 extra github API calls, and you still gotta wait over an hour)

[tekktrik]

I believe this is now a duplicate of #366, correct?