search

w3c / sync-media-pub

Repository of the Synchronized Multimedia for Publications Community Group
http://w3c.github.io/sync-media-pub
Other
17 stars 4 forks source link

player experiment: "custom read aloud" #19

Closed danielweck closed 4 years ago

danielweck commented 5 years ago

Experiment with https://github.com/thephilgray/custom-readaloud-plugin

danielweck commented 5 years ago

Test with this URL: https://raw.githack.com/w3c/sync-media-pub/feature/custom-read-aloud-player/samples/single-document/index.html

marisademeglio commented 5 years ago

Cool! One thing though - did the sync points change in the narration document, or is the player not respecting the clip boundaries? The original version that I posted purposely excluded the audio for "read by for librivox.org".

danielweck commented 5 years ago

Note: both demos (Marisa's own experiment, and this one which uses an existing lib) currently hard-code the CSS "active" classes. These should be declared somewhere so that a playback engine (irrespective of the implementation details) can discover what class names to use.

danielweck commented 5 years ago

Note: this implementation requires a single audio resource, whereas (I think) the sync-media-narration JSON can reference different audio files (?).

danielweck commented 5 years ago

The playback model in this implementation does not seem to be capable of handling clip begin and end. I other words, the injected "play head" is equivalent to clip-begin only, there is an assumption that audio clips are contiguous within a given audio stream.

danielweck commented 5 years ago

Bug? The text highlight only syncs when audio playback reaches the known beginning of an audio fragment. There is stale highlighted text when scrubbing the playback bar directly in the audio controls UI.

danielweck commented 5 years ago

I filed an issue here: https://github.com/thephilgray/custom-readaloud-plugin/issues/21

danielweck commented 5 years ago

I filed another issue: https://github.com/thephilgray/custom-readaloud-plugin/issues/22

danielweck commented 5 years ago

Note how this implementation enables keyboard navigation by setting an ARIA role of button and tabindex on all synchronized elements. This behaviour should probably activated "on demand" to avoid UX issues, i.e. activate specifically during the synch'ed text/audio experience. But otherwise, I like the idea :)

marisademeglio commented 4 years ago

Closing as this would overwrite the existing demo. Suggest putting in a different folder.

danielweck commented 4 years ago

For sure. Let's delete my branch as well to avoid future housekeeping. The code diff is trivial and will be preserved in this PR for posterity anyway :)