developmentseed / bioacoustics-frontend

Frontend code for the Google Bioacoustics project that's used by A2O Search
https://search.acousticobservatory.org/
MIT License
1 stars 0 forks source link

Update/grid card styles #70

Closed LanesGood closed 1 year ago

LanesGood commented 1 year ago

Makes some minor style changes to the grid card style

image

I've also attempted to add a "playing" overlay animated via css to fire when the grid card's audioUrl is playing, but the isPlaying state from the useAudio hook is not firing when this hook is called from the parent results/gridView component.

https://github.com/developmentseed/bioacoustics-frontend/assets/12634024/52b625e3-6c88-4491-a25d-a9950e2b162f

@oliverroick I am able to make this work by removing the results/AudioPlayer component and running all of this component's logic within the parent results/gridView file. Is there a better way to execute this?

oliverroick commented 1 year ago

I am able to make this work by removing the results/AudioPlayer component and running all of this component's logic within the parent results/gridView file. Is there a better way to execute this?

The only way to do this is to lift the audio player state into the ResultCard component and then pass the props to the AudioPlayer component. We can refactor that but I'd suggest to treat this as a feature and separately. There are currently three branches all dealing with results display and the changes in this branch will affect other branches too.

LanesGood commented 1 year ago

As a test, I lifted the entire audioplayer component into the Result Card and was able to get this working.

Interestingly, it still doesn't result in a perfect sync because the audio has to make a network request to get the clip and start playing, while the CSS animation is triggered immediately.

LanesGood commented 1 year ago

One last follow-up: it is possible to sync the css with the audio time playback perfectly by setting width: ${(currentTime / duration) * 100}% if we pull these values from the useAudioPlayer hook. This is a bit jerky.

I also am seeing differences in the network tab with the useAudioPlayer hook being called by default on each card, instead of in the child-component AudioPlayer I'm guessing this is requiring each audio file to be downloaded before any audio can be played, which is not the functionality we want. Is this correct @oliverroick?

oliverroick commented 1 year ago

One last follow-up: it is possible to sync the css with the audio time playback perfectly by setting width: ${(currentTime / duration) * 100}% if we pull these values from the useAudioPlayer hook. This is a bit jerky.

I pushed a change that uses this suggestion, and changed the way currentTime is updated. Instead of waiting for the timeupdate event, which only updates 3 times per second, I used and interval to read the time from the audio element. I've set the interval to 25ms so it updates at ~40frames/second, which makes the progress smoother. We can probably increase the frame rate further but have to be careful because each of these triggers an re-render of the component using the hook.

I also am seeing differences in the network tab with the useAudioPlayer hook being called by default on each card, instead of in the child-component AudioPlayer I'm guessing this is requiring each audio file to be downloaded before any audio can be played, which is not the functionality we want. Is this correct @oliverroick?

I think what you're seeing is the partial download of the meta data for the audio, to the the duration. It doesn't download the full audio file. The full audio is only downloaded when the stream starts.

LanesGood commented 1 year ago

Thanks @oliverroick! Looks great. Marking this ready for review.

I think what I'm noticing on the audio playback is that there is often a delay between pushing "play" on the result clip and hearing the audio feedback. This delay seems more present when the spectrogram images for each card on the page are loading. Probably not worth looking at now as we'll address some of this in the pagination step, but worth noting that it may also need some suspense/lazy handling.