Media Overlays support in ZIP EPUB mode (files not expanded)

[danielweck]

Originally reported by @aadamowski at https://github.com/readium/readium-js/pull/28

Media overlays currently do not work for zipped EPUBs when read using plain browser-based viewer utilizing the readium-js library - that's a known issue that will require non-trivial changes to be solved (due to inherent limitations of a pure JS, pure client side, browser-based environment).

Taking this into account, does anyone still have problems:

• on Safari,
• with reading zipped EPUBs,
• served through the embedded server launched from the Grunt build, listening on port 8080,
• while the same zipped EPUBs work fine on Chrome browser, while accessed from the aforementioned server?

[caasi]

(due to inherent limitations of a pure JS, pure client side, browser-based environment)

Cloud you please give more explanation about those limitations?

If I hope someday one can load and play media overlay in a pure client side environment, where should I start to look at?

[danielweck]

Just to be clear: EPUB3 Media Overlays is fully supported by the ReadiumJS cloud reader, which is a pure client-side implementation. However, loading resources (individual files) from within a zip archive (the EPUB container) is technically feasible on the client side runtime, but this is only acceptable (performance-wise) for small assets, not large audio/video files (because each asset is loaded entirely in memory). The performance degradation is significant for typical MO audio files. Note that it would be possible to "stream" audio/video media on the client side via a Service Worker, and there is in fact an embryo of proof of concept inside the ReadiumJS GitHub repository. There are pros/cons though, as always with any technology.

[danielweck]

Oh, and just to be extra clear: by client-side I obviously mean web browser. A hybrid app that integrates a webview within a host framework would obviously provide adequate mechanisms to extract resources on the fly from a zipped EPUB archive.

[caasi]

Thank you for those details. :D

My friend and I are working on the EPUB version of the bopomofo handbook for teachers. It's a small project paid by the Ministry of Education in Taiwan. He wants to use Media Overlays to read those tonal marks. But we failed with the zipped book.

I am going to read zip_resource_fetcher.js to understand ReadiumJS techonology to see if I can workaround it for my case.

[danielweck]

You will see in readium-js document "fetchers" that when loading a packed / zipped EPUB, all the resources attached to an EPUB HTML file are statically loaded into "binary blobs", which then replace src attributes in the HTML markup (using Blob URIs). This is called "programmatic" fetching in the source code. Normally, the webview loads assets in a much smarter way, especially audio and video media (on-demand, incremental pre-fetching, loading, etc.).

The problem specific to Media Overlays is that audio files are loaded dynamically by the MO playback engine (which code is in readium-shared-js). The audio files are not actually preloaded by the readium-js document "fetchers", simply because HTML documents do not reference these audio files (instead, the SMIL files do). So, to solve this problem, the MO engine would need to query a single audio file URL (on the fly, as defined by the SMIL files), pointing to a resource that is in fact located inside the zipped EPUB container. Because of the aforementioned performance issues, development effort was never spent to implement this functionality.