Closed tfabris closed 5 months ago
First of all, big thank you for the huge work you have done with this! These scripts have been a big help with my own bird box setup. Anyway, do you have plans to continue working with this 'new broadcast and bind the stream to that broadcast' routine?
I'm so glad that the script was helpful to you in the past.
I want to work on it, but I'm not sure when I'll get back to it. There's a secondary factor at work here:
But... The beauty of GitHub is that I should theoretically be able to accept code changes from other people and incorporate it into this script. If you'd like to give it a shot and see if you can get it working under the new YouTube paradigm, I'd love to work with you on that.
@tfabris If you could identify and outline the changes that need to be made, I could implement them and then maintain.
@tfabris If you could identify and outline the changes that need to be made, I could implement them and then maintain.
@dkelson - Thanks so much for the offer. All the information I have is at the top of this thread. It’s a little vague because I don’t have all the knowledge myself. If I knew exactly which code changes to make, I would have already made them.
There are some things there which are just for me, not possible for others to do: for example, moving my personal video feed to a new YouTube channel and updating my personal website code. Clearly you couldn't do those things for me.
The first item on the todo list, that is the one that could be taken on by someone else, and submitted as a pull request: “Rewrite the code to do the necessary steps to create a new broadcast, and bind the stream to that broadcast, each time it is required.” This will require that you analyze and understand the current code behavior, and understand why YouTube’s API changes broke it. There are some links included there which should helpful in understanding why it broke. The code depended on the stream being the same stream all the time, and just needing a slight poke once in a while to keep it going. Now YouTube has a whole new set of requirements about setting up a new stream from scratch each time you want to restart the video playback.
If you want to take this on, I can try to support you where needed. You’ll need your own private YouTube channel, Synology, and security camera to run it on. Once you’ve got something that works, you can do a PR and we’ll review it together.
Let me know if you’re up for it!
@tfabris "You’ll need your own private YouTube channel, Synology, and security camera to run it on"
I have these three things.
This I need to work on wrapping my mind around. Since I'm not the original author, it's a more of an uphill climb but doable. "analyze and understand the current code behavior, and understand why YouTube’s API changes broke it."
@dkelson ,
Indeed, yes, learning the code flow of someone else's code is an uphill climb. But that's always the case when one takes on this kind of a task. It's what happens in all software development (open-source or otherwise), and I'd go so far as to say that it's the hardest aspect of software development. I've never seen anyone teaching a class on how to do it.
On the good side, my code is heavily commented with as much information as I could possibly include. And I'm also available to answer questions: if you choose to accept this mission, feel free to contact me by private email once you start running into things that you don't understand. My email address is at the Vixy & Tony web site.
Thanks!
I have switched to an ISP which doesn't have a data cap. So this script could theoretically be made to work again. Currently investigating whether or not I can do this.
Issues which are currently presenting themselves (more to come):
Status so far:
TO DO:
Current status:
To do:
There appears to still be something wrong here.
Yesterday there should have been three segments, the last segment running from about 7pm to about 8pm (since the prior segment runnng up to 7pm was four hours long and got split at the four hour mark). The Synology log showed the four hour split mark as normal, but that last third segment never showed up on the list of videos for the channel. Not in the Uploads playlist nor the CrowCam Archives playlist. I don't know if there was even a stream up at that point or not.
Then the following morning, there was a similar situation with the 6:35am stream at sunrise. The log said it was starting the stream but nothing was showing up on the youtube channel.
The CrowCam.sh is not throwing any errors or other messages to the Synology log, it just had the 6:35am stream startup message and that's it.
I went to YouTube's Live Dashboard stream at https://www.youtube.com/live_dashboard and it showed that it was expecting some stream data to reach it. There were no errors or anything, it just had the spinny icon saying that I needed to connect my streaming software to send data. But looking at the Synology "Live Broadcast" feature, it was on (and the log said it had been turned on at 6:35am), and the stream key was correct (the Synology one matched the one in the YouTube Live Dashboard).
I'm going to investigate by running the script on the Synology and looking at its shell debug output.
There are no error messages when running the CrowCam.sh script at the shell prompt. It sees that the Synology "live broadcast" feature is on as it should be, it retrieves the bound stream ID correctly, successfully checks that the stream key is correct, makes sure there are no network glitches, and says everything is just hunky-dory, no need to bounce the stream.
From the Synology perspective, everything is fine.
TO DO:
The code already looks for things like "videoIngestionStarved" but that's not happening in this case, the YouTube API is not returning that particular error or any other error, even though that's exactly the situation it's encountering (there seems to be nothing sending video data to it at the moment).
I have noticed that when I use the API to query the live stream, I see this response (notice how it says healthStatus is Good even though there is no actual stream to be healthy, wow that's fscked up):
{
"kind": "youtube#liveStreamListResponse",
"etag": "asdf1234asdf1234asdf1234asdf1234",
"pageInfo": {
"totalResults": 0,
"resultsPerPage": 5
},
"items": [
{
"kind": "youtube#liveStream",
"etag": "asdf1234asdf1234asdf1234asdf1234",
"id": "asdf1234asdf1234asdf1234asdf1234asdf1234",
"status": {
"streamStatus": "active",
"healthStatus": {
"status": "good"
}
}
}
]
}
However according to the API documentation of the LiveStreams Resource, there is more data in other fields if I query for them, for example, the stream resolution and frame rate is in the "CDN" field. Trying this: Bounce the Synology live broadcast, see if it makes the stream come back, and if it does, see if any of those other things can be a good coalmine canary (since the YouTube API is lying in its "status" field).
Nope, the LiveStreams resource seems identical. Even when I query for everything, id, snippet, cdn, and status, it has the same output. So the LiveStreams resource is not the correct resource to look for.
However if you query the LiveBroadcasts resource, I do see what I hope is a good query. Not certain about this one but we'll see.
"status": {
"lifeCycleStatus": "live",
"privacyStatus": "public",
"recordingStatus": "recording",
"madeForKids": false,
"selfDeclaredMadeForKids": false
},
When I was reproducing the problem I could see that the values:
To do: Update the Test_Stream function to also check for these, and bounce the stream if those things aren't what is expected.
Note: There is also a tantalizing value "boundStreamLastUpdateTimeMs" in the "contentDetails" part...
"contentDetails": {
"boundStreamId": "UxIqdJJpXBnjaEy5yq1wZQ1705457117285100",
"boundStreamLastUpdateTimeMs": "2024-03-18T18:39:32Z",
"monitorStream": {
However this is not a useful value to query. It does not change, even when the stream is live and working. This is just the moment that the ID changed, not the moment of the video pixels were updated.
The detection of lifeCycleStatus and recordingStatus is a good indicator. However, if lifeCycleStatus is "complete" then we have to figure out how to start a new video and go live. I tried doing the simplest thing which is to perform an INSERT operation:
'https://youtube.googleapis.com/youtube/v3/liveBroadcasts?part=snippet%2Cstatus&key=[YOUR_API_KEY]' \
--header 'Authorization: Bearer [YOUR_ACCESS_TOKEN]' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data '{"snippet":{"title":"CrowCam","scheduledStartTime":"2024-03-18T20:55:00Z"},"status":{"privacyStatus":"public"}}' \
--compressed
This didn't quite work the way I'd hoped. It created an "UPCOMING" video which I now have to figure out how to transition to being live. Investigating...
Aha, OK we do have to do the whole rigamarole described in the lifecycle of a broadcast document. Working on that.
With d7fa902, a new stream is finally created at sunrise and when the stream is being split.
To do:
I believe that 5f5280e - Final fix for issue #69 ? is the final fix for issue #69.
This morning I awoke to a situation where, despite there being nothing else that had gone wrong with the stream, it was down, and the stream was in the state recordingStatus: recorded lifeCycleStatus: complete.
That looks like a really good litmus test for the need to rebuild the livestream. So I added code to do precisely that, at the place where that detection occurs. When I put it into production, it worked exactly as expected immediately. Does that mean it works? (You shut up, Betteridge.)
I had originally thought that I would need to do a fancier check, trying a short bounce first and if that doesn't work then try a long bounce. Luckily the log messages about recordingStatus and lifeCycleStatus showed me that there was a perfectly good way to check it directly without having to do anything fancy on my side.
Closing this bug finally! If anything else comes up, it will probably need to be a separate bug anyway.
PS: @caso and @dkelson - Feel free to try the code in its current state. I'm still working on some existing issues, so keep a lookout for new code updates in the near future.
The CrowCam code depended on the YouTube features "Default Stream" and "Persistent Stream", which they deprecated on approximately 2020-09-10. The code no longer works correctly.
To do:
Helpful links: