tfabris
commented
5 months ago With checkin 1f98a91 I have reduced the number of hysteresis loops in Test_Stream from 4 to 2 and reduced the pause between tests from 15 to 7. That means that if the stream is bad, instead of waiting a whole minute to bounce it, it's now only waiting 14 seconds.
This still isn't perfect though, because the program doesn't re-query the stream throughout the entire run of the CrowCam.sh program. Instead the program currently works like this:
- Do all the timing-related stuff first (sunrise, sunset, hourly quick bounce, etc.).
- Quick check the stream state. If bad, bounce it (or rebuild it if lifeCycleStatus says so)
- For five minutes (well, four minutes and change), keep looping and testing the network ping.
- Check the stream state again. If bad, bounce it (or rebuild it if lifeCycleStatus says so)
Checkin 1f98a91 improves the hysteresis at steps 2 and 4, but still leaves a five minute "hole" during step 3 where the stream isn't being queried at all. A worst-case scenario would look like this:
- Step 2 passes fine, the stream state is checked and it's fine.
- YouTube fscked up the stream just after step 2, but my network was fine (that's the state I saw on April 1st which made me write this bug)
- Then there could be a five minute period where the stream is bad, but it's not being checked. Five minutes might be too long and still make the livestream die, regardless of my hysteresis settings.
Options to fix this:
- Do nothing. Maybe the changes I already made to the hysteresis of Test_Stream fixed it, but even if they didn't, CrowCam still "worked" on April 1st, it just created a new livestream from scratch several times that day when it maybe didn't need to.
- I could place a Test_Stream inside the 5-minute network test loop instead of once before and once after. However. This is something that previously existed already, and I deliberately deleted it way back at issue #9, and I've got code comments telling me that I deleted it on purpose. Some reasons not to Test_Stream during every network ping test:
- Eats up much more YouTube API quota.
- Slows down the network test loop by an unpredictable amount, since the Youtube API calls might not return quickly, making the overall network test loop run much slower than expected (this was my original reason for deleting this)
- Perhaps I could compromise and do Test_Stream just once at the halfway point of the network test loop, rather than every loop. That would reduce my failure surface from 5 minutes to 2.5 minutes, and would only double my API usage instead of increasing it by an order of magnitude.
- Or, without changing the code logic, I could reduce the number of network tests by half, and then just schedule the script to run twice as frequently. That would have a similar outcome as sticking a stream test in the middle of the network test loop. It would consume more YouTube API quota though, because there are some other things in the script which also consume quota besides Test_Stream. i don't think I like this idea as much as the halfway test idea.
- Consider changing the code which creates new streams and see if you can remove the "Auto Stop" flag. If this can be done, then you'll need to add more complex code which ensures that you "stop" the stream with an API call anytime you end a broadcast on purpose, discover a new stream needs to be created, or split the stream on purpose.
To do in the meantime:
- Watch the logs and see if the April 1st error rears its ugly head again, and see if your changes to the Test_Stream hysteresis was enough to fix it.
- See if you can find documentation about exactly how long the data can stop squirting before YouTube kills your livestream out from under you (this may be two different amounts, depending on whether Auto Stop is enabled or not). UPDATE: It's "about one minute" if the Live Broadcast contentDetails.enableAutoStop is set to true. Don't know what happens if it's set to false, might be worth finding out.
I observed a behavior several times on 2024-04-01 where:
Relevant log entries: