search

VikeLabs / lecshare

A React application with typescript implementation: Making lectures more accessible for students with auditory, visual, or learning impairments; and Improving learning outcomes for students in lecture-based courses.
https://vikelabs.github.io/lecshare/
GNU General Public License v3.0
5 stars 3 forks source link

Slow audio loading. #38

Open aomi opened 4 years ago

aomi commented 4 years ago

For a full sized lecture (ie 50 minutes or so), it takes a while for the audio to load and be ready for playback. A simple solution to this is to enable the HTML5 option in the Howler.js lib.

However enabling this kills the loading all together and currently there is a bug in Howler.js. There appears to be a workaround if the server can respond to partial content requests.

https://github.com/goldfire/howler.js/issues/1110

https://github.com/goldfire/howler.js/issues/1180

aomi commented 4 years ago

A potential solution. https://github.com/AnthumChris/opus-stream-decoder

Some readings about the issue. https://github.com/WebAudio/web-audio-api/issues/337 https://discourse.wicg.io/t/webcodecs-proposal/3662 https://github.com/WebAudio/web-audio-api-v2/issues/61

guest271314 commented 4 years ago

An AudioWorkletNode can be substituted for createBufferSource() and createSourceBuffer(), see https://plnkr.co/edit/nECtUZ?preview where a 291MB WAV file is fetch()ed parsed to Float32Arrays and streamed to audio output, an alternative approach using a 70+MB WAV file spliced into N files and streamed https://github.com/guest271314/AudioWorkletStream. There is the potential for issues with glitches at *nix at Chromium https://bugs.chromium.org/p/chromium/issues/detail?id=825823.

Note, latencyHint can affect Callback Buffer Size and Render Capacity of AudioContext instance. https://github.com/WebAudio/web-audio-api-v2/issues/70. Whether Cache disabled is checked or not can also affect audio output if AudioWorkletProcessor is being used, which may or may not be related to *nix/PulseAudio callback timing (priority) issue.

guest271314 commented 4 years ago

Given the duration of the audio is known, if animation of words on screen is synchronized to audio output, that synchronization is not explicitly necessary. Animations can run separately from audio due to total duration known. Element.animate() can be used separately from audio timing and should nonetheless (be configured to) progress at the same rate.