tfabris / CrowCam

A set of Bash scripts to control and maintain a YouTube live cam from a Synology NAS.
GNU General Public License v3.0
4 stars 3 forks source link

Investigate using the YouTube API to determine if the live stream is up or not. #26

Closed tfabris closed 5 years ago

tfabris commented 5 years ago

Currently I have code in CrowCam.sh to detect if the YouTube stream is up, and if not, tries to bounce the stream.

Detecting if the stream is up is a separate thing from detecting if the network is up. Both require a check. Network check is more serious and is done more frequently, several times per script run. Stream check is only done once per run of the script.

Right now, the code uses Youtube-DL as its tool for checking stream upness. This is inefficient because that's not really what that tool was designed to do. There could be a more efficient check perhaps, if I could leverage the YouTube API to perform a stream upness check. Details of the APIs required are in this general vicinity: https://developers.google.com/youtube/v3/live/docs/liveBroadcasts/list

This has the potential for missing a bad-stream situation, a place where the stream is down but YouTube thinks the stream is up. An example of this situation is issue #23 - In that situation, YouTube thinks the stream is fine but it shows a spinny death icon instead of showing the actual live stream. Well, the thing is, YouTube-DL doesn't detect that problem either, so it's no help in that situation either. So likely a method for using the YouTube API to check upness is going to be just as good as using YouTube-DL to check upness.

tfabris commented 5 years ago

Here is some working code which can display the YouTube API variable "boundStreamLastUpdateTimeMs". Unfortunately, this does not quite work for our purposes. I was hoping that it could tell me when we were encountering a "hung stream", but it doesn't seem like there's an API for that.

It does successfully obtain the stream key (very useful for the separate issue #24) but doesn't solve our upness detection problem.

Instead, I need to search for a simple query of whether or not YouTube thinks the stream is live or not. There should be an API for that which I can leverage.

#------------------------------------------------------------------------------
# Function: Convert a number of seconds duration into HH:MM format.
# 
# Parameters: $1 = integer of the number of seconds since midnight
#
# Returns: Formatted time string in HH:MM format.
#
# Technique gleaned from: https://unix.stackexchange.com/a/27014
#------------------------------------------------------------------------------
SecondsToTime()
{
  # If the number of seconds is a negative number, then the event is in the
  # past, so flip the number to positive so that we can show the HH:MM time
  # without a bunch of ugly "-" signs in the output. If the number is 0 or
  # positive, then use the number as-is.
  if [ $1 -ge 0 ]
  then
    # Use the number as-is if it's 0 or positive.
    T=$1
  else
    # If the number is negative, flip it to positive by multiplying by -1.
    T=$((-1*$1))
  fi

  # Perform the calculations and formatting based on the code example.
  local D=$((T/60/60/24))
  local H=$((T/60/60%24))
  local M=$((T/60%60))
  local S=$((T%60))

  # Return the resulting string to the code that called us.
  printf "%d days %02d:%02d\n" $D $H $M
}

# Get the current live broadcast information details as a prelude to obtaining
# the live stream's current/active "Secret Key" for the default live broadcast
# video on my YouTube channel.
# Some help here: https://stackoverflow.com/a/35329439/3621748
# Best help was here: https://stackoverflow.com/a/40422459/3621748
# Requesting the "part=contentDetails" gets us some variables including
# "boundStreamId". The boundStreamId is needed for obtaining the stream's
# secret key. Note that you could also request things like "part=snippet"
# instead of the "part=contendDetails" to get more info, such as the video
# description text, if you wanted. You can request a bunch of pieces of text
# simultaneously by requesting them all together separated by commas, like
# this: part=id,snippet,contentDetails,status
curlUrl="https://www.googleapis.com/youtube/v3/liveBroadcasts?part=contentDetails&broadcastType=persistent&mine=true&access_token=$accessToken"
liveBroadcastOutput=""
liveBroadcastOutput=$( curl -s $curlUrl )

# Debugging output. Only needed if you run into a nasty bug here.
# Leave deactivated most of the time.
# logMessage "dbg" "Live Broadcast output information: $liveBroadcastOutput"

# Extract the boundStreamId which is needed in order to find the secret key.
boundStreamId=""
boundStreamId=$(echo $liveBroadcastOutput | sed 's/"boundStreamId"/\'$'\n&/g' | grep -m 1 "boundStreamId" | cut -d '"' -f4)
logMessage "dbg" "boundStreamId: $boundStreamId"

# Extract the timestamp to help in determining if the stream has hung up.
# Note: This does not actually help because it describes when the stream's
# details changed, such as the text description, not when the video stream
# itself last got valid data.
boundStreamLastUpdateTimeMs=""
boundStreamLastUpdateTimeMs=$(echo $liveBroadcastOutput | sed 's/"boundStreamLastUpdateTimeMs"/\'$'\n&/g' | grep -m 1 "boundStreamLastUpdateTimeMs" | cut -d '"' -f4)
logMessage "dbg" "boundStreamLastUpdateTimeMs: $boundStreamLastUpdateTimeMs"

# Calculate difference between system current date and the date that was
# given by the API query of the last update time. Warning: I expect these lines
# to fail on MacOS due to Mac having a different "date" command.

# I think these convert to local time automatically. For instance if I put in
# 2019-05-08T15:26:11.276Z (which is 3:26pm UTC) then I get 8:26am output.
# For debugging the various output options for the time stamp, change this
# variable here and force it to the time stamp you want to test:
#     boundStreamLastUpdateTimeMs="2019-05-08T21:24:11.276Z"
lastUpdatePrettyString=$( date -d "$boundStreamLastUpdateTimeMs" )
lastUpdateDateTimeSeconds=$( date -d "$boundStreamLastUpdateTimeMs" +%s )

# So, use local time for your local side of the calculations.
currentDateTimePrettyString=$( date )
currentDateTimeSeconds=$( date +%s )

# Get the difference between the two times.
differenceDateTimeSeconds=$(( $currentDateTimeSeconds - $lastUpdateDateTimeSeconds ))
differencePrettyString=$( SecondsToTime $differenceDateTimeSeconds )

logMessage "dbg" "Date/Time of last YouTube Stream update:  $lastUpdateDateTimeSeconds seconds, $lastUpdatePrettyString"
logMessage "dbg" "Date/Time of current system:              $currentDateTimeSeconds seconds, $currentDateTimePrettyString"
logMessage "dbg" "Stream was last edited:                   $differenceDateTimeSeconds seconds ago, aka $differencePrettyString ago"

# Obtain the stream key from the live stream details, now that we have the 
# variable "boundStreamId" which is the thing that's required to get the key.
# We can query for part=id,snippet,cdn,contentDetails,status, but "cdn" is the
# one we really want, which contains the special key that we're looking for.
# Note: if querying for a specific stream id, you have to remove "mine=true",
# you can only query for "id=xxxxx" or query for "mine=true", not both.
curlUrl="https://www.googleapis.com/youtube/v3/liveStreams?part=cdn&id=$boundStreamId&access_token=$accessToken"
liveStreamsOutput=""
liveStreamsOutput=$( curl -s $curlUrl )

# Debugging output. Only needed if you run into a nasty bug here.
# Leave deactivated most of the time.
# logMessage "dbg" "Live Streams output information: $liveStreamsOutput"

# Obtain the secret stream key that is currently being used by my broadcast.
# Note: This variable is named a bit confusingly. It is called "streamName" in
# the YouTube API, and identified as "Stream Name/Key" on the YouTube stream
# management web page. But it's not really a "name" per se, it's really a
# secret key. We need this key, but we're using the same name as the YouTube
# API variable name to identify it.
streamName=""
streamName=$(echo $liveStreamsOutput | sed 's/"streamName"/\'$'\n&/g' | grep -m 1 "streamName" | cut -d '"' -f4)
logMessage "dbg" "Secret stream key for this broadcast (aka streamName): $streamName"

exit 0
tfabris commented 5 years ago

This might have what we're looking for: liveBroadcasts?part=status

curlUrl="https://www.googleapis.com/youtube/v3/liveBroadcasts?part=status&broadcastType=persistent&mine=true&access_token=$accessToken"
liveBroadcastOutput=""
liveBroadcastOutput=$( curl -s $curlUrl )

It contains a response that looks like this. Maybe "lifeCycleStatus" is what we want?

  "status": {
    "lifeCycleStatus": "live",
    "privacyStatus": "public",
    "recordingStatus": "recording"
   },

There is also LiveStreams which contains this good stuff which might be even better: https://developers.google.com/youtube/v3/live/docs/liveStreams#resource

"status": {
    "streamStatus": string,
    "healthStatus": {
      "status": string,
      "lastUpdateTimeSeconds": unsigned long,
      "configurationIssues": [
        {
          "type": string,
          "severity": string,
          "reason": string,
          "description": string
        }
      ]
    }
  },
tfabris commented 5 years ago

My investigation is done. I have working code in place in production, and I'm letting it live there a while to see if it's as reliable as YouTube-DL or better.

There was supposed to be a variable called "lastUpdateTimeSeconds" for the healthStatus which might have been helpful, but it's not being sent by the API despite being listed in the dox.

In the meantime my code is using the "status" field of the "status" part of the "LiveStreams" resource, and also the "healthStatus" field, as described here: https://developers.google.com/youtube/v3/live/docs/liveStreams#resource

Those seem to be the only two things that give me the status of the actual stream. I can't find anything else.

tfabris commented 5 years ago

Turns out the code does indeed bust the quota. After having this run in production for a day, the quota got busted at about 7:30pm and now I have to wait until midnight PST to get it to reset.

Need to rethink the queries. I'll open a separate bug about this.

tfabris commented 5 years ago

Quota problem is solved, and the code which uses YouTube-DL to check stream upness is experimentally removed, in the most recent checkin.