search

unfoldingWord-dev / uw-android

Android unfoldingWord app
Other
9 stars 4 forks source link

Audio Not Synchronized with FRAMES #127

Closed jag3773 closed 9 years ago

jag3773 commented 9 years ago

Full screencast: https://app.testfairy.com/projects/1370651-unfoldingword/builds/1816391/sessions/1/?accessToken=Ou8Q/z7fb4YiR7EG6NJabcZ5zhQ

The images do not automatically slide as the audio is played (e.g. timing markers are not being used).

GottaGetSwifty commented 9 years ago

Is there any documentation about how this is supposed to work?

bknatterud commented 9 years ago

What are you unclear on? The frames of OBS advance as the audio is played.

bknatterud commented 9 years ago

There are markers in the audio file.

bknatterud commented 9 years ago

Upon some further investigation I am seeing some issues with using markers in the MP3 files.

  1. Have not yet found a library in Java that can retrieve marker information that looks good. IDV3 information yes, lot's out there for that but not markers so much.
  2. If we are able to get the markers out the marker information in the MP3 data contains a startTime and duration but the values are in frames not seconds or milliseconds. My initial research seems to suggest that it may not be possible to convert those frames to a reliable timestamp and it appears that when audio is playing we are not able to easily get the current frame of media being played.

Perhaps somebody else on the team has already cracked this issue on mobile. But, right now I'm not sure if reading in the markers from the MP3 file will really work.

An alternative might be to run the MP3 files through a process on the server that converts those markers to a JSON file that has matching frame numbers and timestamps.

And yet another approach would be to actually split each frame up into a separate MP3 file. Which, might actually be the fastest way to make this work. In the research I was doing it appears that in Adobe Audition you can export your audio in chunks based on your markers. So I don't think generating the files would be a huge undertaking.

I'm surprised that utilizing these markers is not easier.

@jag3773 please let me know what you are thinking on this. My thought is let's break the MP3 files for OBS up into one MP3 file per frame. Clean and simple.

bknatterud commented 9 years ago

Also, since this data is really Adobe's XMP data these resources could be helpful in utilizing it if we go the route of extracting it.

http://www.adobe.com/devnet/xmp.html

Which brings me to one concern about using markers. I was under the impression that most audio editing applications allow you to add these markers and that the spec for adding this metadata is universal. But now I'm not so sure.