Creating Supervised Classifier for Novel Species

[Cesarmedina123]

Describe the bug First off, thank you very much for updating DeepSqueak v3.01. I have high hopes to use this software moving forward, but running into some technical challenges when creating a supervised network.

I am trying to create a supervised neural network in DeepSqueak v3.01 using MatLab 2020b for detecting zebra finch calls based on modified call categories from Badwal et al., 2018, but I am running into some issues. I am using .wav files and importing them into DeepSqueak then saving the files after drawing boxes around the calls I see (this is not explicitly stated in the wiki). Why can't we just use the .wav files to create the training images for training the unsupervised network?

When creating training images from the detection files to create the unsupervised network first (as recommended in the wiki), I am unable to identify parameters that can properly detect calls (i.e., get a good classifier that detects 50%) of calls. I eventually used the following parameters:

(except that I use an "Image Length (s)" of .5, 1 or 2 to get a good bout length) to create a poorly performing unsupervised neural network on a subset of training data (I detect less than 1% of calls, but very poorly, the boxes are drawn over a part of the call always). I then run the new neural network on a testing data set. However, it barely detects any calls (as indicated above). I therefore modified the boxes and drew new boxes around calls that I saw. I also labeled these calls with labels that I would like to train a supervised network on. However, after doing this for some 400+ calls, I cannot train the supervised network since I require training images. Why are training images required and how do i create training images that preserve the call classifications I assigned manually? Also, when I copy the detection and .wav files over to the "Detections" and "Audio" folder for DeepSqueak, I get the following error (in bold) when selecting detection files for creating a supervised classifier (unfortunately, I cannot include the .mat files here).

**Dot indexing is not supported for variables of this type.

Error in TrainSupervisedClassifier_Callback (line 16) wind=options.spectrogram.windowsize;

Error in gui_mainfcn (line 95) feval(varargin{:});

Error in DeepSqueak (line 29) gui_mainfcn(gui_State, varargin{:});

Error in matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)DeepSqueak('TrainSupervisedClassifier_Callback',hObject,eventdata,guidata(hObject)) Error while evaluating Menu Callback.

Error using audioread>readaudio (line 130) The filename specified was not found in the MATLAB path.

Error in audioread (line 123) [y, Fs] = readaudio (filename, range, datatype);

Error in squeakData/AudioSamples (line 100) obj.StoredSamples = audioread(obj.audiodata.Filename, [obj.AudioStartSample, obj.AudioStopSample]);

Error in CreateFocusSpectrogram (line 61) audio = audioReader.AudioSamples(box(1), box(1) + box(3));

Error in CreateClusteringData (line 118) [I,wind,noverlap,nfft,rate,box,~,~,~,~,pow] = CreateFocusSpectrogram(Calls(i,:), handles, true, p.Results.spectrogramOptions, audioReader);

Error in TrainSupervisedClassifier_Callback (line 9) [ClusteringData, Class, options.freqRange, options.maxDuration, options.spectrogram] = CreateClusteringData(handles, 'scale_duration', true, 'fixed_frequency', true);

Error in gui_mainfcn (line 95) feval(varargin{:});

Error in DeepSqueak (line 29) gui_mainfcn(gui_State, varargin{:});

Error in matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)DeepSqueak('TrainSupervisedClassifier_Callback',hObject,eventdata,guidata(hObject)) Error while evaluating Menu Callback.**

I used the following label names: "Harmonic.Unmodulated", "Harmonic.Modulated", "Noisy", "Mixed", etc

Any help would be helpful since a detection network for this species does not exist. I suspect that the spectrogram is not being saved in the appropriate variable or that the labels are causing an issue (should not include a period).

[Cesarmedina123]

Edit: Some of the issues were that there was a period in the label name for one of the categories and the corresponding audio file could not be found in the MatLab path. I will continue updating once I create a good (un)supervised network, but keep this open here since I would still like to know why a .wav file cannot be used for creating an unsupervised network (having to crete the detection file first) and why we cannot manually draw windows for making detection training images that could be used to create a supervised detection network?

[DrCoffey]

Hey @Cesarmedina123

Training a good detection network is the first step and can be a bit tricky. I don't have much experience with Zebra Finch calls, so it is hard to give specific advice on the training step. Would you mind sharing and audio recording and a corresponding detection file so I can look over the calls and give you better advice? Everything can be de-identified, and I won't share the recordings with anyone.

-Kevin

[Cesarmedina123]

Hola @DrCoffey

What would be the best way to send it to you? I can't share .wav or .mat files via Github?

Saludos,

Cesar

[DrCoffey]

Hey @Cesarmedina123 I fixed an issue with the supervised classifier code today. Just a heads up so you can update your code.