search

sul-dlss / sul-embed

An oEmbed Service for Stanford University Libraries
Other
19 stars 6 forks source link

Sidebar transcript tab does not load in Safari #2068

Closed alundgard closed 5 months ago

alundgard commented 6 months ago

In Safari only, the Sidebar transcript tab does not load.

PURL Examples:

Image

Console logs:

In Firefox and Chrome, the Sidebar transcript tab loads as expected. Image

hudajkhan commented 6 months ago

The transcript isn't loading in Firefox for me either, although I do see it in Chrome. Also the video itself is not loading either.

jcoyne commented 6 months ago

The transcript isn't loading in Firefox for me either, although I do see it in Chrome. Also the video itself is not loading either.

It works fine for me in Firefox.

hudajkhan commented 6 months ago

That is good to hear.

hudajkhan commented 6 months ago

Oddly enough it is now working for me in Firefox. And also in Safari although I do see the text track cross origin error.

Adding notes:

https://purl.stanford.edu/qf378nj5000

-Transcript loads intermittently in Firefox i.e. worked fine in private window even after hard refresh. Worked fine in regular window the first time, but stopped after hard refresh.

hudajkhan commented 6 months ago

Error examples (from browser stack trial using windows and Firefox).:

Loading module from https://ga.jspm.io/npm:@videojs/vhs-utils@3.0.5/es/ was blocked because of a disallowed MIME type (text/html).
iframe
Loading module from https://ga.jspm.io/npm:three@0.157.0/ was blocked because of a disallowed MIME type (text/html).
iframe
Loading module from https://ga.jspm.io/npm:@lit/reactive-element@1.6.3/decorators/ was blocked because of a disallowed MIME type (text/html).
iframe
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://ga.jspm.io/npm:@videojs/vhs-utils@3.0.5/es/. (Reason: CORS request did not succeed). Status code: (null).

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://ga.jspm.io/npm:three@0.157.0/. (Reason: CORS request did not succeed). Status code: (null).

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://ga.jspm.io/npm:@lit/reactive-element@1.6.3/decorators/. (Reason: CORS request did not succeed). Status code: (null).

From Firefox on Mac:

GET
https://ga.jspm.io/npm:@videojs/vhs-utils@3.0.5/es/

GET
https://ga.jspm.io/npm:@lit/reactive-element@1.6.3/decorators/

GET
https://ga.jspm.io/npm:three@0.157.0/

Loading module from “https://ga.jspm.io/npm:@videojs/vhs-utils@3.0.5/es/” was blocked because of a disallowed MIME type (“text/html”).
[iframe](https://embed.stanford.edu/iframe?url=https://purl.stanford.edu/qf378nj5000&_v=1704721322)
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://ga.jspm.io/npm:@videojs/vhs-utils@3.0.5/es/. (Reason: CORS request did not succeed). Status code: (null).

Loading module from “https://ga.jspm.io/npm:three@0.157.0/” was blocked because of a disallowed MIME type (“text/html”).
[iframe](https://embed.stanford.edu/iframe?url=https://purl.stanford.edu/qf378nj5000&_v=1704721322)
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://ga.jspm.io/npm:three@0.157.0/. (Reason: CORS request did not succeed). Status code: (null).

Loading module from “https://ga.jspm.io/npm:@lit/reactive-element@1.6.3/decorators/” was blocked because of a disallowed MIME type (“text/html”).
[iframe](https://embed.stanford.edu/iframe?url=https://purl.stanford.edu/qf378nj5000&_v=1704721322)
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://ga.jspm.io/npm:@lit/reactive-element@1.6.3/decorators/. (Reason: CORS request did not succeed). Status code: (null).

From Safari on (my local) Mac, for "Women at Stanford" which does load the transcript

[Error] Failed to load resource: the server responded with a status of 404 () (npm:three@0.157.0, line 0)
[Error] Failed to load resource: the server responded with a status of 404 () (es, line 0)
[Error] Failed to load resource: the server responded with a status of 404 () (decorators, line 0)
[Error] Failed to load resource: the server responded with a status of 403 (Forbidden) (playlist.m3u8, line 0)
jcoyne commented 6 months ago

@hudajkhan those errors are unrelated. That is due to import-map 2.0.0. The problem occurred before that was merged in.

hudajkhan commented 6 months ago

Thanks @jcoyne ! I was trying to look at what errors are occurring when it doesn't seem to work and when it does. I was wondering if this warning 

"VIDEOJS:"WARN:"
"Text Tracks are being loaded from another origin but the crossorigin attribute isn't used.
This may prevent text tracks from loading."

is the issue, but I get this warning when the page works for me in Safari. Will keep digging.

hudajkhan commented 6 months ago

For reference, the documentation includes this section: https://videojs.com/guides/text-tracks/#text-tracks-from-another-domain.

hudajkhan commented 6 months ago

After analysis and help from @alundgard and @andrewjbtw , we realized that this error is a different manifestation of #2062 with the added wrinkle of OS level caption settings.

In summary: (a) In Safari, videojs text tracks do not have cues loaded for any track that is not already selected in the video player. Our transcript code retrieves all the cues for the different caption tracks and loads them when the video player has dispatched events indicating some level of readiness. Unfortunately, in the case of Safari, while the code can see caption tracks even if they are not selected in the video player itself, the cues are not available.
(b) There are OS settings on Macs that will determine if a caption is selected by default in the video player. Alan and Andrew tested out the settings and found when the Accessibility "Prefer closed captions and SDH" setting was turned off, the video player had no captions selected by default. In that case, the transcript would not load. When they turned the setting on, a caption file was selected by default and the transcript would load in the sidebar.

I have been searching for a mechanism whereby we could programmatically select the caption files and/or extract all the cues by changing track modes to "showing" or "hidden" instead of "disabled". Unfortunately, I haven't found a solution yet.

hudajkhan commented 6 months ago

One approach to try for single tracks is to see if adding "default" to the track will help select that track even if the OS settings have "prefer closed captions and SDH" turned off.

hudajkhan commented 6 months ago

What I tried and what seemed to work in Safari:

  1. Adding a "default" attribute for a track in what we generate does appear to have that track selected by default in Safari - even if the "prefer closed captions and SDH" setting is off. This allows the cues to be available and the transcript to load.
  2. In Safari: Unfortunately, setting the track mode to "showing" or "hidden" (changing the setting from "disabled"), does not allow the track cues length to return as non-zero. Interestingly, the console logs are actually able to see the cues and list them but the length is still zero. This stackoverflow discussion seems to be saying something similar: https://stackoverflow.com/questions/23067294/videojs-texttracks-cues-empty. The method "show" does not appear to be valid for a track at this point. If there was a way to trigger the track to load through an event, or similar, then it might work. We'd have to be careful not to actually have the video player itself select the language for the triggered event, if this approach is even possible.

I'll be setting up a PR with (1) to address the single track issue in Safari. I wasn't able to find a solution yet for #2062 .

hudajkhan commented 6 months ago

The only other approach I can think of that may work for Safari is to bypass videojs and read in the caption using WebVTT directly (for the transcript). I would have to explore that approach.

hudajkhan commented 6 months ago

Testing out the pull request branch on staging:

Testing this out on safari for an item with multilingual captions https://sul-purl-stage.stanford.edu/wm783fb6812: When the video first loads, English is selected in the video caption list. The English transcript is visible in the sidebar, and the highlighting and auto-scrolling work correctly.

When Russian is selected in the video, the highlighting no longer works in the transcript. This is an additional error in the multilingual caption situation.

For a video like https://sul-purl-stage.stanford.edu/pf759xf5671, there is only one language and the transcript loads fine. When captions are turned to off though in the video itself, the transcript highlighting and autoscrolling no longer work.