PublishedAt / OriginallyAvailableAt date discrepancy

[micahmo]

Hi @ZeroQI, back again! I hope this hasn't been covered already (I scoured the issues), but it seems like there is occasionally a discrepancy between the publishedAt date returned by the items in the playlist API call vs the direct video info call.

(BTW, I'm using your latest code from rewrite.)

Take this video, for example: LlKnEgPEGmA

The playlist API call returns this date.

https://www.googleapis.com/youtube/v3/playlistItems?part=snippet&maxResults=50&playlistId=PLt5zc-HxINkpAr9Ri_qJtzklB9YK2Eunh&key=KEY
"publishedAt": "2018-12-05T03:33:42Z",

The video API call returns this date.

https://www.googleapis.com/youtube/v3/search?&maxResults=1&part=snippet&q=LlKnEgPEGmA&key=KEY
"publishedAt": "2015-05-04T18:30:00Z",

I was able to "fix" the issue by going directly to the video info (just like the movie code). See this snippet with my additions.

# videoId in Playlist/channel
videoId = Dict(video, 'id', 'videoId') or Dict(video, 'snippet', 'resourceId', 'videoId')
if videoId and videoId in filename:
  episode.title                   = filterInvalidXMLChars(Dict(video, 'snippet', 'title'       ));     Log.Info('[ ] title:        {}'.format(Dict(video, 'snippet', 'title'       )))
  episode.summary                 = filterInvalidXMLChars(Dict(video, 'snippet', 'description' ));     Log.Info('[ ] description:  {}'.format(Dict(video, 'snippet', 'description' ).replace('\n', '. ')))
  # micahmo added below
  json_video_details = json_load(YOUTUBE_json_video_details.format(videoId, Prefs['YouTube-Agent_youtube_api_key']) )['items'][0]
  episode.originally_available_at = Datetime.ParseDate(json_video_details['snippet']['publishedAt']).date();  Log.Info('[ ] publishedAt:  {}'.format(json_video_details['snippet']['publishedAt']))
  # micahmo added above
  # micahmo removed: #episode.originally_available_at = Datetime.ParseDate(Dict(video, 'snippet', 'publishedAt')).date();  Log.Info('[ ] publishedAt:  {}'.format(Dict(video, 'snippet', 'publishedAt' )))
  thumb                           = Dict(video, 'snippet', 'thumbnails', 'standard', 'url') or Dict(video, 'snippet', 'thumbnails', 'high', 'url') or Dict(video, 'snippet', 'thumbnails', 'medium', 'url') or Dict(video, 'snippet', 'thumbnails', 'default', 'url')
  if thumb and thumb not in episode.thumbs:  episode.thumbs[thumb] = Proxy.Media(HTTP.Request(thumb).content, sort_order=1);                                Log.Info('[ ] thumbnail:    {}'.format(thumb))
  Log.Info('[ ] channelTitle: {}'.format(Dict(video, 'snippet', 'channelTitle')))
  break

However, I'm assuming you don't want to hit the individual video API for every video again, so I'm not sure what the permanent fix is. For me, I'm nowhere near my API quote, so I don't mind adding this, but I'm not sure if others can say the same.

Maybe it can be optional for those with a low quote that also have issues with video dates?

[bcc32]

The PlaylistItem property snippet.publishedAt refers to the time the item was added to the playlist. Perhaps you want contentDetails.videoPublishedAt instead? see here

[micahmo]

@bcc32 Thanks for the response, you're exactly right! It's confusing because snippet.publishedAt for a single video is the real published date.

Looks like it's easy to add contentDetails to the request...

https://www.googleapis.com/youtube/v3/playlistItems?part=snippet,contentDetails& . . . 

I might put together a PR for this, unless @ZeroQI beats me to it. :-)

[micahmo]

Hey @ZeroQI,

Sorry to bother you again! Looks like this issue came back when rewrite was merged to master. Commit 92c576c overwrote the changes from PR #75.

[ZeroQI]

My bad, updated while on work computer, didn't refresh agent on home computer...

[micahmo]

No worries, I hate to bother you after all the help you've been!

It looks like the latest commit c6cae63 has a ton of changes like everything got duplicated.

I think the fix for this bug should only be two lines.

episode.originally_available_at = Datetime.ParseDate(Dict(video, 'contentDetails', 'videoPublishedAt')).date();  Log.Info('[ ] publishedAt:  {}'.format(Dict(video, 'contentDetails', 'videoPublishedAt' )))
. . .
YOUTUBE_PLAYLIST_ITEMS = YOUTUBE_API_BASE_URL + 'playlistItems?part=snippet,contentDetails&maxResults=50&playlistId={}&key={}'

[ZeroQI]

pasted without ctrl+A prior seemingly... Thanks for spotting

[micahmo]

Thanks, all good now. Appreciate your help and time, as always!