Experience with delay - Githubissues

AddictedCS / soundfingerprinting

Open source audio fingerprinting in .NET. An efficient algorithm for acoustic fingerprinting written purely in C#.

https://emysound.com

MIT License

946 stars 189 forks source link

Experience with delay #146

Closed smxthereisonlyone closed 4 years ago

smxthereisonlyone commented 4 years ago

We'd like to put audio triggers into a video stream and then detect those on iOS & Android to synchronize. Is that possible and with what delay?

AddictedCS commented 4 years ago

What kind of triggers? Acoustic triggers of type "a specific audio signal in a specific frequency range", or just any regular audio content?

smxthereisonlyone commented 4 years ago

We'd do 20 minutes of video stream with 20 minutes of audio. The video starts and goes for a few minutes. Now, imagine a user opens an app that includes sound fingerprinting, then sound fingerprinting starts and we'd like to know the exact timecode of the video. Is that possible?

AddictedCS commented 4 years ago

This is achievable.

You need to fingerprint and store your streaming audio first.
When the second app starts, fingerprint 10-15 seconds chunks and query fingerprints storage.
Check the results. If Confidence is > 0.5, check TrackMatchStartsAt, this should give you the position where the query matched in the fingerprinted stream.
Seek to TrackMatchStartsAt + QueryLength. This should sync both apps.