search

mytakedotorg / mtdo

The code and tooling which runs mytake.org
https://mytake.org
GNU Affero General Public License v3.0
16 stars 3 forks source link

Audio drifts in our fact player, but not in YouTube #239

Open nedtwigg opened 4 years ago

nedtwigg commented 4 years ago

In our fact explorer, if you drag the "now playing" bar to the far right, towards the end of the video, then the audio is badly out of sync with the video. If you open them in YouTube, this does not happen.

Videos that have the drift problem:

Videos that don't:

This is a total guess, but I wonder if it has anything to do with this field:

https://github.com/mytakedotorg/mytakedotorg/blob/103b92b2d280a092704d1d18fad65c5929947042/presidential-debates/1960-10-13.json#L9

Perhaps for the videos that happen to be exactly x.0 seconds long, this works fine, but for the videos that happen to be x.5 seconds long, the audio ends up being 0.5 seconds out of sync towards the end. I haven't tested that hypothesis against the data above, it's just a guess.

WebsByTodd commented 4 years ago

I'm struggling to reproduce this consistently. When I first stated debugging, I was able to see the audio drift on our site but not on YouTube. After a few tries though, the problem went away and I couldn't bring it back. I changed no code.

Interesting discussion: https://support.google.com/youtube/thread/16931671?hl=en

The javascript here is pretty straightforward, when we scrub the now playing bar (either click on the bar or release a drag) we tell the youtube player directly using its own API to seek to a time. There are no independent audio vs video controls exposed through their API.

Any thoughts? I'm happy to hop on a screen-share chat today to brainstorm.

nedtwigg commented 4 years ago

We're probably going to have to settle for how it is until we get a YouTube expert. I tried downloading the streams using youtube-dl, and then examining them with ffprobe.

It's not a slam dunk by any means, but the most interesting thing I could find was that the "DRIFTS" videos have the same 'seconds' place for audio and video duration, whereas "WORKS" has different seconds for the audio and video. It seems unlikely that would be the cause.

DRIFTS '2016-10-19 - Presidential Debate - Clinton, Trump (3 of 3)-dg8f1jwzezk.mkv':
Video DURATION        : 01:33:47.821000000
Audio DURATION        : 01:33:47.913000000
DRIFTS '1960-10-21 - Presidential Debate - Kennedy, Nixon (4 of 4)-lzeZXCPjH2U.mkv':
Video DURATION        : 00:58:54.897000000
Audio DURATION        : 00:58:54.959000000
WORKS '1976-10-06 - Presidential Debate - Jimmy E Carter, Gerald R Ford (2 of 3)-SLy5fq1b4jA.mkv':
Video DURATION        : 01:27:03.951000000
Audio DURATION        : 01:27:04.188000000
WORKS '1980-10-28 - Presidential Debate - Carter, Reagan (1 of 1)-a1J1TkWCLhs.webm'
Video DURATION        : 01:34:51.384000000
Audio DURATION        : 01:34:52.001000000

Full data:

DRIFTS: Input #0, matroska,webm, from '2016-10-19 - Presidential Debate - Clinton, Trump (3 of 3)-dg8f1jwzezk.mkv':
  Metadata:
    HTTP://YOUTUBE.COM/STREAMING/OTF/DURATIONS/112015: Segment-Count: 1054 
                    : Segment-Durations-Ms: 5339(r=1052),6206, 
                    :  
                    : 
    ENCODER         : Lavf58.29.100
  Duration: 01:33:47.91, start: 0.000000, bitrate: 1054 kb/s
    Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv, bt709/unknown/unknown), 1280x720, SAR 1:1 DAR 16:9, 29.97 fps, 29.97 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      DURATION        : 01:33:47.821000000
    Stream #0:1: Audio: aac (LC), 44100 Hz, stereo, fltp (default)
    Metadata:
      HANDLER_NAME    : SoundHandler
      DURATION        : 01:33:47.913000000

DRIFTS: Input #0, matroska,webm, from '1960-10-21 - Presidential Debate - Kennedy, Nixon (4 of 4)-lzeZXCPjH2U.mkv':
  Metadata:
    HTTP://YOUTUBE.COM/STREAMING/OTF/DURATIONS/112015: Segment-Count: 662 
                    : Segment-Durations-Ms: 5339(r=660),6039, 
                    :  
                    : 
    ENCODER         : Lavf58.29.100
  Duration: 00:58:54.96, start: 0.000000, bitrate: 867 kb/s
    Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv, bt709/unknown/unknown), 1280x720, SAR 1:1 DAR 16:9, 29.97 fps, 29.97 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      DURATION        : 00:58:54.897000000
    Stream #0:1(eng): Audio: aac (LC), 44100 Hz, stereo, fltp (default)
    Metadata:
      HANDLER_NAME    : ISO Media file produced by Google Inc.
      DURATION        : 00:58:54.959000000

WORKS: Input #0, matroska,webm, from '1976-10-06 - Presidential Debate - Jimmy E Carter, Gerald R Ford (2 of 3)-SLy5fq1b4jA.mkv':
  Metadata:
    HTTP://YOUTUBE.COM/STREAMING/OTF/DURATIONS/112015: Segment-Count: 979 
                    : Segment-Durations-Ms: 5339(r=977),2736, 
                    :  
                    : 
    ENCODER         : Lavf58.29.100
  Duration: 01:27:04.19, start: 0.000000, bitrate: 284 kb/s
    Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv), 456x360, SAR 1:1 DAR 19:15, 29.97 fps, 29.97 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      DURATION        : 01:27:03.951000000
    Stream #0:1: Audio: aac (LC), 44100 Hz, stereo, fltp (default)
    Metadata:
      HANDLER_NAME    : SoundHandler
      DURATION        : 01:27:04.188000000

WORKS: Input #0, matroska,webm, from '1980-10-28 - Presidential Debate - Carter, Reagan (1 of 1)-a1J1TkWCLhs.webm':
  Metadata:
    HTTP://YOUTUBE.COM/STREAMING/OTF/DURATIONS/112015: Segment-Count: 1066 
                    : Segment-Durations-Ms: 5339(r=1064),5706, 
                    :  
                    : 
    ENCODER         : Lavf58.29.100
  Duration: 01:34:52.00, start: -0.007000, bitrate: 319 kb/s
    Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv), 480x360, SAR 1:1 DAR 4:3, 29.97 fps, 29.97 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      DURATION        : 01:34:51.384000000
    Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 01:34:52.001000000