Interactive Play Button / Tracking

[hglanz]

@michalg04 My R twitter friends came through when I asked a little bit about this. Here are some things for you to check out and report back on :)

First, a few links about HTML and JavaScript: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/audio https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/currentTime

....though these might not be new.

What looks REALLY promising: https://github.com/Athospd/wavesurfer

@mschrothglanz check out the wavesurfer github link just above and scroll down to some of his example clips. This looks AMAZING!

[hglanz]

@michalg04 @mschrothglanz I actually wonder now, how much of the app we might want to change in light of the wavesurfer functionality. There appear to be a lot of really cool things that we already does that it does too.

So, perhaps follow up task/goal for @michalg04 ........the example clips on that github show some cool widgets and functionality for use with a spectrogram. Our app clearly does more than just a spectrogram. Some questions/things to explore:

• Can wavesurfer do more than just a spectrogram? i.e. can it do spectrum, etc.
• Is wavesurfer's spectrogram different/better/worse than ours? If better, does using it screw up our joint functionality with the spectrum and oscillogram?
• Anything other thoughts?

I'm sorry we're just finding out about wavesurfer now, but it doesn't seem to totally obsolete a bunch of stuff or anything :)

[mschrothcp]

@hglanz @michalg04 this is very exciting!!

[michalg04]

@hglanz @mschrothglanz I've been able to implement the audio play oscillogram under our current plots (excuse my really weirdly skewed screen)

I can see this being an extra feature we add but I am not sure if we can get it to be interactive with the other graphs. This is an HTML widget object, and it cannot be converted to ggplot object to become interactive. We can maybe get the coordinates of each segment we highlight in the play area to be reflected in the graphs above, but I'm not entirely sure how to do that at the moment. I'll keep working on it. A side note - I haven't been able to republish the app. Accessing it now gives an error that says I need to check the logs, but the logs don't indicate any failure or problem. I pushed the code to GitHub so you can run it through R-Studio.

[mschrothcp]

@hglanz are you able to determine the publishing problem? @michalg04 are you saying that wavesurfer's spectrogram is not interactive? Under the "annotator" part of the github Hunter shared it seemed that you could interact with the spectrogram.

[michalg04]

@mschrothglanz wavesurfer is somewhat interactive, it just isn't compatible with our graphs so we probably won't be able to get that sound oscillogram to interact with our spectrogram or spectrum. You can still select an area and such but the zoom the way it works on our graphs won't be possible.

[michalg04]

@hglanz @mschrothglanz I finally fixed the publishing problem! App should be updated now. Sorry it took me so long. Let me know what you think of the wavesurfer, it's just an added feature for now. I haven't figured out how to involve it in our graphs.

[mschrothcp]

@michalg04 @hglanz Hello! So I finally found some time to take a closer look at the app. I just have a couple questions/issues: 1) When I open the page, I'm not seeing what we use to see. Instead I'm seeing this: Eventually, after clicking some buttons, I can get back to our original template.

2) I love that we can play it and see the current position of the sound wave within the oscillogram. However, it would be nice if we could see it in the spectrogram as well. The spectrogram that the new function provides isn't colored and doesn't have a y-axis which is also problematic. Is this something that we can change? Are we able to get the playback button on the spectrogram as well?

3) The ability to annotate is really nice but, obviously would be much better within the spectrogram. Our primary graph should be the spectrogram since that is what we work with most often.

4) I don't quite understand the minimap or timeline buttons.

5) The dolphin file seems to crash the app. As we mentioned before @hglanz, getting the power to work this app is needed.

What is everyone else's thoughts?

[michalg04]

@mschrothglanz thank you for your feedback!

1. I noticed that problem too when I republished the app. If I open the app with Safari and not Chrome it looks the way it did before. It's also totally normal when I run it from RStudio. Not sure why it's browser sensitive now. Nothing in the code changed that part so it's one of those Shiny problems I'm not sure how to explain at the moment and my only guess is that we might be overloading the app which is creating these bugs. 2 & 3. I'm not able to change much about the functionality of the wavesurfer at the moment, but I am still researching it.
2. I didn't see anything in the documentation that explained those so I removed those functions for now. I also removed the spectrogram since it's not very nice looking at the moment. I also think these features slow down the app further so not having them for now is better in my opinion.

Sorry I don't have better news at the moment!

[mschrothcp]

@michalg04 That's okay. Is there anything I can do on my end? Or perhaps @hglanz has some ideas....