vidify / old-audiosync

First implementation of the audio synchronization feature for Vidify, now obsolete
GNU Lesser General Public License v3.0
17 stars 3 forks source link

Not precise enough #6

Closed marioortizmanero closed 4 years ago

marioortizmanero commented 4 years ago

The problem

This module works quite well when the recording starts as soon as a new song starts. But in a real situation, the recording starts after the following happens in Vidify:

  1. The API detects a new song: some APIs can have a delay of ~1000ms to notice this and may be straight up unusable. This is partly why Linux is only supported for now, since MPRIS can asynchronously recieve the changes and is more precise.
  2. The API notifies the GUI to start a new video with a Qt signal.
  3. The GUI receives the signal and runs the function to play the video. 4. The URL of the audio to be downloaded is obtained with youtube-dl.
  4. The get_lag function is finally called.
  5. The lag is set in on_audiosync_done 6. The player ALSO has lag. It's usually about 2000ms. And it depends on whether you're using VLC or Mpv (the latter has less lag).

VLC doesn't work well because it uses a large GoP size by default and the seeking isn't precise enough for the synchronization.

Ideas:

Original

img

Shifted

img

In this picture, the green shifted part at the beginning is the ending of the original track data. If at least it was empty, the confidence matching would be better. You can tell the shifting was actually quite precise, but the coefficient (0.060655 in this case) is affected by the initial part.

Original

Original

Shifted

Shifted

To avoid having too little precision, MIN_CONFIDENCE is raised to something like 0.95 (previously 0.75).

marioortizmanero commented 4 years ago

Precision without noise has improved enough since this issue was created. The problem now is precision when there's noise. I'll open a new issue for that.