Discuss Orcasound architecture schematics

[scottveirs]

Spring 2021 schematics generated by Paul Cretu for community discussion:

Existing architecture (still accurate Aug 2022)

Proposed architecture (development in 2023+)

[scottveirs]

There's a lot to discuss here, but this winter (2022-23) is looking like a good time to start responding to some of the proposed architectural changes that @paulcretu suggested last year. Here's a short-list of opportunities/tasks I see on the horizon:

• [ ] Start streaming FLAC in addition to HLS audio data, possibly at sample rates of 96 or 192, rather than only 48 kHz.
  • [x] As an experiment with the UW MSDS students, start streaming FLAC from only Port Townsend (around 1/1/23)
• [ ] For HLS segments, shift from standard filename format to one with a date-time stamp
• [x] Add hydrophone locations: Point Robinson, Sunset Bay, north San Juan Channel
• [ ] Leverage Karan's GSoC code to ingest external data from OOI hydrophones on Oregon coast
• [ ] Continue discussion with HALLO group of real time inference on Canadian hydrophones (BC Hydrophone Network members, DFO, ONC, SMRU, etc.)
• [ ] Shift from current S3 buckets to sponsored open data registry buckets (2023-2025, renewable for 2-year periods)
• [x] Transition from Azure credits to pay-as-you-go, then quantify per-node costs for 1-6 months
• [ ] Continue user-centered design of Observations and Shipnoise presentation layers (e.g. dashboards)
• [ ] Strategize about integration of Admin layers (Orcasound admin UI, OrcaHello moderation UI, Dataplicity or other monitoring tools)
• [ ] Integrations with other data providers and end users of real time location data via SSEMMI (Salish Sea Ecosystem Monitoring and Modeling Infrastructure), e.g. via the prototype distributed data cooperative deployed at acartia.io](https://acartia.io), including diversified and customizable notification methods

Discussing all of them in advance is wise as many of these evolutions have the potential to enhance, improve, and/or break things in the existing architecture!

[dbainj1]

FLAC will be a problem for Point Robinson until the internet connection there gets improved. Higher sampling rates will be valuable where the hydrophones support it, but I expect we'll be building out the system with inexpensive hydrophones in the near future, so increasing the sample rate won't help at those sites. A research grade setup should have a 500 kHZ sample rate, although lower sample rates can add some additional functions that we don't have at 48kHz.

Can we get a hydrophone in at MAST this fall?

I like the idea of shifting to a filename format that includes date, time and location. Since hydrophones have limited lifespans and different frequency response curves, it might be helpful to include hydrophone id in the filename (saves us from having to check other sources for when a hydrophone was replaced).

We also have the Oceans Initiative data which are pre-recorded, but still need to be run through the system.

Ship noise data will be more meaningful with lossless compression. It would also be good to incorporate calibration curves, if available, into processing noise data before it is presented.

Monitoring system uptime will be a valuable addition. Having automatic notification of a problem will allow us to address them sooner than waiting a while and wondering why notifications have not been received.

[scottveirs]

As of early January, Port Townsend is streaming FLAC files at 48kHz sample rate, as well as the normal HLS segments. The FLAC files can be browsed here via Quilt -- https://open.quiltdata.com/b/archive-orcasound-net/tree/rpi_port_townsend/