Categorizing Signals in Real-Time

[DrCoffey]

We would like DeepSqueak to be able to perform real-time categorization of detected calls.

• [ ] Apply DeepSqueak classification methods real-time during recording.
• [ ] Display call cluster in DeepSqueak app in the “Label” section

[vijayiyer05]

@DrCoffey, I'm having a good discussion with the proposed collaborator Marvin now. One question that has come up is whether the second item applies to labeled classifiers (e.g. YOLO) and cluster classifiers (e.g. k-means, VAE) alike? Or is it the second one for clustering only?

We can discuss more at a kick-off meeting, but wanted to capture this first question now.

[DrCoffey]

Currently we don't use the main detection networks (YOLO) for classification. Those are only trained to discriminate a vocalization from the background. There is no reason this couldn't be done in theory, but different labs have different ways they like to classify calls, so we keep main detection networks general.

The intention for real-time classification would be to apply the post-hoc classifiers (k-means, VAE, or even CNN) to the individual calls as they are detected.

[DrCoffey]

Also, the current release of DeepSqueak doesn't have any of these classifiers built in (because they are generally lab specific), but I can create some very quickly to test with.

[vijayiyer05]

Thanks @DrCoffey for clarifying. I got the dichotomies mixed up! I was thinking about how for some species classification works (e.g. CNN) and for others clustering is needed (e.g. k-means or VAE). My assumption is that both tasks apply to both cases, but we briefly wondered if one applied to each since one says classify and the other cluster.

[ARTUR551]

@DrCoffey Hi, are there any general purpose built in classifiers now? Where can I find them? I'm struggling classifying manually and could really use a template to start with. Best, A.