Cloud Reader, Media Overlays: wrong sync after clicking text

[pettarin]

I cloned the git repo and followed the instructions to build the Cloud Reader. I did not perform any code modification.

I copied the dist/cloud-reader directory to a Web server, and added an (unzipped) reflowable EPUB 3 with Media Overlays. (To be precise, this one: http://www.readbeyond.it/samples/bierce01.epub )

EDIT 2015-11-30: I put it online here: http://www.readbeyond.it/readium/?epub=epub_content/bierce01

The issue is the following:

1. open the eBook, navigate to the first chapter associated with a SMIL
2. start the Media Overlays rendition
3. while the rendition is active, go a few screens forward, and click on an arbitrary sentence to restart the audio from it
4. when the audio restarts --- after buffering the new position, I guess --- the text highlighting is no longer in sync with the audio. Usually, the audio is just slightly off (<1s), but other times it is significantly off. Sometimes the timing is correct.

The same EPUB (hence, the same SMIL) shows "correct" behavior when viewed on Readium Chrome Extension or on Menestrello app.

It looks like a buffering / < audio > element / Javascript problem to me, but I do not know enough to debug myself. Is this a known issue?

[danielweck]

Hello @pettarin ,

This seems to be a Chrome-specific HTML5 audio bug. I am able to reproduce the incorrect playback offset (testing with the EPUB you linked to), using Readium "cloud reader" hosted on my localhost HTTP server, as well as with the Chrome extension (which serves app files and content resources directly via local filesystem).

I am however unable to reproduce this behaviour when running the exact same code inside Electron (which I believe uses a different Chromium build than the Chrome web browser itself). I suspect this would work fine in Apache Cordova too (with or without CrossWalk). I am also unable to reproduce the bad audio seeking on Internet Explorer, Edge and Firefox.

I have only tested on Windows, as I do not have my OSX machine within reach right now.

Are you on Linux? (I am surprised that the Readium Chrome app works fine in your tests)

PS: I saw this kind of HTML5 audio seeking bug before, it appeared to be related to the MP3 encoding of the audio files. 100% reproducible with certain EPUBs, totally okay with most Media Overlays I test.

As a general note: it has been very tricky to create buffering+seeking code that works reliably across all target platforms ... HTML5 audio is inconsistent depending on the actual backend (iOS QuickTime vs. Android MediaServer etc.) and browser / webview integration.

[pettarin]

Yes, I was testing under Linux (Debian Jessie) with Chrome 46.0.2490.86 from Google's repo. (My current Iceweasel (Firefox 38.2.1), which comes from the Debian's repo, does not reproduce MP3 audio files at all, so I did not test with it.)

I also tested with a Lenovo Android 4.4 tablet with Chrome, same problem. On Safari on iOS, I need to double tap in order to trigger a "single click" event. I was not able to trigger a click on text (?), so I was not able to check for the same problem there; using the "previous/next fragment" buttons keeps the audio/highlighting in sync.

I know well that working these details involving buffering, audio, and timing across all platforms is really tricky. Thank you for checking/confirming this issue, I will try different encodings for the audio files, and see which one achieves the broadest support.

EDIT 2015-11-30: I put it online here: http://www.readbeyond.it/readium/?epub=epub_content/bierce01

[pettarin]

Update: I re-encoded all the audio files inside the EPUB files at http://www.readbeyond.it/ebooks.html using ffmpeg/lame.

On the positive side, if I download the entire EPUB and load it in the latest Readium Chrome Extension, the issue seems solved.

On the other hand, the issue is still present in the cloud reader, I believe due to the buffering of the audio file, but I guess there is nothing more you or I can do about it, rather than suggesting to the end users to wait a few seconds before starting the MO playback. (Actually, a loading bar or another GUI element would help.)

[paopic]

Hello everyone,I made an EPUB 3 with synchronized audio using the mp3 extension, create the smil file and there is no way to paint each of the words synchronously.

Now I change the extension to .m4a and it works perfect, but when I want to try it on android does not work the audio.

Here I show you an example of smil.

<?xml version="1.0" encoding="utf-8"?>

I probe to do it in 0:00:00 and did not work. Or is it directly android does not support this epub format?

Thank you for your help in advance. Regards

(Forgive my English, I'm Argentine.)

[danielweck]

Hello @paopic your SMIL example is missing (not attached to your comment).

What Readium-based reading system are your using to test your EPUB?

Daniel

[paopic]

Ohhh sentimos! `

Estoy usando Epub Reader para Android, en teléfonos inteligentes borde S7

Gracias, estoy muy desorientado.` 005.txt

[danielweck]

If your e-reader (reading system) is not based on Readium, then I am afraid this GitHub issue tracker is not the most appropriate place to ask for help. I would suggest trying the IDPF forum: http://idpf.org/forums/epub-3

If however your Android e-reader app is Readium-based, then could you please provide a link to it on the Google PlayStore? Thank you!

[paopic]

https://play.google.com/store/apps/details?id=com.graphilos.epub

Thank you very much for your attention

[danielweck]

I don't think this e-reader is built with Readium. I would suggest contacting the app vendor directly. Sorry I cannot help. Kind regards, Daniel

[Praveen188]

In cloud reader media play buttons are not working. Please provide the solution Thanks.

[danielweck]

Hello @Praveen188 please specify your browser details, the version/git-revision of the cloud reader you are testing with, which publication you are using, and whether you are attempting to load the packed/zipped EPUB, or the exploded/unzipped EPUB. Thanks :)

[Praveen188]

Hi @danielweck we are using google chrome Version 74.0.3729.108 (Official Build) (64-bit); readium-js-viewer downloaded from git development branch; We are using zipped folder EPUB. Thanks,

[danielweck]

Thank you. Please try with unzipped / unpacked EPUB. It should work. Thanks!

[Praveen188]

Thank you. It's working