Sound Visualizations Ideas and Tools

[valentina-s]

This issue serves as a place to discuss potential sound visualization ideas and tools (there may be better home for this?). Visualizations can fall into different categories:

• visualizations aimed at integrated with the website (which may require specific tools and alignment with UX teams), stand alone visualizations which could be built with any tool and serve as future building components or part of a viz gallery
• Spectrogram visualizations with integrated sound: real-time vs archived
• Spectrogram visualization with integrated sound: with/without option to annotate
• Annotation visualizations: from humans vs from machines
• Annotation visualizations: with/without sound
• With/without maps
• Summary Statistics
• Post Analysis Visualizations
• Education focused
• ...

Non-exaustive List of Visualizations and Tools

• Integrated within Orcasound AI tools:

  • OrcaAL: front end with 3D spectrogram (annotation option), based on Chrome Experiments 3D Spectrogram
  • Orca-Hello Moderator Dashboard

• Stand Alone JavaScript Spectrograms:

  • Audio Annotator (stand alone .js annotator)
  • Realtime Spectrogram Visualization with WebAudio

• Visualization of a library of sounds:

  • Bird Sound Experiments Grid
  • @wetdog had a simplified version of the above visualization
  • Embedding Projector

• Viz tools from Canadian team? @scottveirs?

• Python widget: a spectrogram widget with a scroller can be made with holoviz

• Visualizing Long Streams with Pattern Radio

• BirdNet Live Stream, repo

• General Viz Libraries (could be good for annotations)

  • Altair: a Python library creating declarative visualizations for the web
  • VegaLite: grammar for web visualizations: Altair outputs are in Vega specification
  • D3: lower level library for web viz: simple plots might be easier with VegaLite, but more specific visualizations may require D3
  • Plotly: supporting Python, R, javascript, etc
  • Mapping
    • https://kepler.gl/
    • https://leafletjs.com/
    • ...

Other References:

• GSOC issue
• Trello Board Issue

[rodrickcalvin]

@valentina-s thanks for the resources

[scottveirs]

There may be additional inspiration and good precedents found in this Orcasound Trello board of bioacoustic analysis approaches and tools: https://trello.com/b/vrDhZeGc/orcasound-bioacoustic-analysis-approaches

[rodrickcalvin]

Wow, Thank you @scottveirs digging in