Text highlight sync only occurs when audio playback reaches declared playhead / "clip begin" timestamp

[danielweck]

Hello, we are currently prototyping with different implementations, and I created a small demo using version 0.2.0 lib from this Pull Request https://github.com/thephilgray/custom-readaloud-plugin/pull/20 (note that I also tried 0.1.6 from the master branch, the bug exists there too)

To reproduce, scrub the playback bar in the native audio element's UI controls, and see how the text highlight is only repositioned when the player's current-time reaches the declared "playhead" ("clip begin", in SMIL parlance): https://raw.githack.com/w3c/sync-media-pub/feature/custom-read-aloud-player/samples/single-document/index.html

[danielweck]

I think that this problem is less obvious with shorter audio clips, such as in the official demo: https://codepen.io/phillipgray/pen/bOmeed

[thephilgray]

@danielweck, thanks for the issues. Sorry for the delayed response. Somehow, I missed the notifications and didn't see these until just now.

So, for example, if the first "data-playhead" is set at 2.1s and the next one is set at 2.5s, and you start the audio at 2.4s, you won't see the first highlight.

this.times is an object for which the keys are the start times and the values are the corresponding HTML elements with the data-playhead attribute (via querySelectorAll in the constructor).

One possible solution would be to implement something like this:

// replicate the class instance for demonstration
var instance = {
    audioClipEnd: 5,
    times: {
        2.1: 'html element for 2.1-2.5',
        2.5: 'html element for 2.5-3.3',
        3.3: 'html element 3.3-end'
    }
};

function findHighlightedElement(currentTime) {

    // get the keys from this.times as an array and sort them from least to greatest
    var keys = Object.keys(this.times).sort((a,b) => a-b);

    // determine the last key
    var isLastKey = key => key === keys[keys.length - 1];

    var keyOfElementToHighlight = 

    // find the key of the element to highlight
    keys.find((key, i) => 

    // the current time should be greater than or equal to it
    currentTime >= key 

    // the current time should also be less than the next key after it
    && currentTime < (isLastKey(key) ? this.audioClipEnd : keys[i + 1]));

    // return the HTMLElement for assigning a highlighting class or attribute
    return this.times[keyOfElementToHighlight];
}

// called within _onTimeUpdate
findHighlightedElement.call(instance, 2.4); // returns "html element for 2.1-2.5"

// ....only update DOM if the highlighted element is different than the previous value

Not sure if this is the best way to handle, but it's the first thing that comes to mind.