Orcasound maintains a hydrophone network in Puget Sound (near Seattle, WA, USA, Northeast Pacific). Killer whales (aka orcas) often swim by these hydrophones (underwater microphones) and vocalize with a wide range of calls.
Through annual Microsoft hackathons since 2019 and with the volunteer efforts of many heroic Orcasound open source contributors, we have trained and continue to refine a deep learning model to find these calls in live hydrophone audio. The model is at the core of the real time inference system we call OrcaHello which aims to help recover the endangered population of Southern Resident Killer Whale (SRKW) - the iconic orcas that frequently seek salmon in Puget Sound, and also range annually from California to Alaska. OrcaHello is a part of ai4orcas.net, an International effort to apply cutting-edge artificial intelligence to orca conservation.
Learn more about OrcaHello via:
This repository contains the implementations for the following components that make up the OrcaHello real time inference system:
The diagram below describes the flow of data through OrcaHello and the technologies used.
As of September, 2022, the data flow steps include:
Each overlapping 2-second data segment is classified as a whale call or / not. Shown below is a 1-minute segment of hydrophone audio visualized as a spectrogram with whale calls detected by the model (delineated by white boundaries).
When whale activity is detected by the model, it sends an email to our Moderators who are killer whale experts (bioacousticians).
Once they receive this notification, they can visit the public Moderator Portal shown below to confirm or reject model detections, and to annotate each candidate.
Most importantly, they confirm whether or not the whale call was emitted by an endangered Southern Resident Killer Whale (SRKW). If a SRKW is confirmed, notifications are sent to subscribers, like this email message (2022 example):
You can contribute by
To contribute a pull request for a specific subsystem, please read the corresponding contributing guidelines and READMEs.
Project Page - contains information about the system and a brief history of the project.