Difficulty training network to new species

[severinejag]

Training Clips.zip Describe the bug In trying to train a new neural network in the updated Deepsqueak version to be able to recognize zebra vocalizations. However, after isolating individual vocalizations using RavenPro, generating .mat files for the manually isolated vocalizations, and going through the instructions listed in the documentation for creating a new Network Training Images, I keep getting the following error messages:

Warning: Variable 'Calls' not found.

In loadCallfile (line 2) In create_training_images_Callback (line 43) In gui_mainfcn (line 95) In DeepSqueak (line 30) In matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)DeepSqueak('create_training_images_Callback',hObject,eventdata,guidata(hObject)) Undefined function or variable 'Calls'.

Error in loadCallfile (line 4) if isstruct(Calls); Calls = struct2table(Calls, 'AsArray', true); end

Error in create_training_images_Callback (line 43) Calls = loadCallfile([trainingpath trainingdata{k}]);

Error in gui_mainfcn (line 95) feval(varargin{:});

Error in DeepSqueak (line 30) gui_mainfcn(gui_State, varargin{:});

Error in matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)DeepSqueak('create_training_images_Callback',hObject,eventdata,guidata(hObject)) Error while evaluating Menu Callback.

Expected behavior I am not sure if because zebra vocalizations are not tonal and much lower than USVs that the program is not able to recognize the vocalizations as calls.

Additional context MS Windows

Thank you in advance for any help you may be able to offer.

[DrCoffey]

Hey Severine,

We just uploaded a fix that should work for you. You will need to re-import your calls from Raven.

I imported the file you attached and was able to create training images and train a detection network. There were only 80 calls so the network isn't perfect but it is really not too bad.

Tips for training:

1. Use bouts (.5s worked for us)
2. Use augmented duplicates ( 1 or 2)
3. Spectrogram settings for creating training images: (These can be tweaked to improve image quality)

windowsize = 0.01 noverlap = 0.005 nfft = 0.02

4. Use MATLAB 2018a for training
5. If you use bouts, you can only create images for 1 audio file at a time, but you can choose all the training tables for the actual training.

Let me know if you have any issues!

-Kevin

Zebra_Network.zip

[DrCoffey]

Hey Severine,

I am just checking in to see if you had any luck with my suggestions. If all went well I am going to close this issue.

Thanks, Kevin

[severinejag]

Hello Kevin,

I have not been able to try out the changes because I am encountering difficulties obtaining MATLAB 2018a through my institution. I am running the most recent MATLAB, and so I wanted to try it on the version you recommended above.

I will let you know if it works once I am able to run it in the right version.

Severine

[DrCoffey]

Ok no problem. If you are signed into mathworks.com you should be able to get 2018a with this link:

https://www.mathworks.com/downloads/web_downloads/download_release?release=R2018a

and then just put your licence info in when you install it.

-Kevin

On Tue, Apr 9, 2019 at 4:59 PM severinejag notifications@github.com wrote:

Hello Kevin,

I have not been able to try out the changes because I am encountering difficulties obtaining MATLAB 2018a through my institution. I am running the most recent MATLAB, and so I wanted to try it on the version you recommended above.

I will let you know if it works once I am able to run it in the right version.

Severine

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DrCoffey/DeepSqueak/issues/39#issuecomment-481482811, or mute the thread https://github.com/notifications/unsubscribe-auth/AlxTE3lNKS5CUvfA2PA1e-vTJmqCk8mEks5vfSkggaJpZM4ce7Gk .

-- Kevin Coffey, Ph.D.

Senior Fellow University of Washington

Harborview Medical Center 325 9th Ave Seattle, WA 98104 USA

mrcoffey [at] uw.edu mrcoffey@uw.edu (860) 874 5659

[severinejag]

Hello,

I finally got the License Key for MATLAB 2018a and did just do a test run with the Zebra Network made with the 80 calls just to make sure everything was running (it would not on MATLAB 2018b). It did detect calls, though I notice in the window comes up when I load calls that I do not see them, though I see them on the smaller image on the side. I suspect this is because the range is so small for the zebra vocalizations (I set it at 1-10kHz).

I also notice when I select "play call" that I just hear white noise-- am I supposed to hear a playback of that particular call that has been identified?

Also, would you be able to break down what you did to import calls to train the network into steps? I suspect there may be an element of user error on my end for creating new training images. Should I be grabbing individual calls from Raven, making a .mat file of them, and then trying to use that associated new .mat file? Or is there a way to take the selection table generated from a Raven file where each call's location is marked along with whatever spectral features are relevant and import that along with the whole audio file?

Thank you again so much for you aid in this issue.

[DrCoffey]

Hey Severine,

You need to change your display range (Tools -> Change Display Range). Set it to 0 - 10 kHz.

Then you need to change playback rate (Tools -> Change Playback Rate).

To create the training data you need to import the raven files (File -> Import From Raven) Then choose the raven log, and corresponding audio files. The resulting .mat files will be placed in your detections folder. Then use those .mat files to create training images (Tools -> Network Training -> Create Detection Network Training Images). Choose 1 .mat at a time so you can create "bout" images.

Use bouts (.5s worked for us)

Use augmented duplicates ( 1 or 2)

Spectrogram settings for creating training images: (These were a guess on my part to make nice images and could be modified, whatever settings are normal for zebra spectrograms).

windowsize = 0.01 noverlap = 0.005 nfft = 0.02

You will then be prompted to choose the corresponding audio file. The images and training table will be generated and placed in a "Training" folder in the main DeepSqueak Directory. Repeat this for all of your .mat files.

Then you can train the network with the corresponding image tables (Tools -> Network Training -> Train Detection Network).

Sorry for how complex this is, we really wanted to maintain fleaxability when training new networks, so the GUI has a bit of a learning curve.

-Kevin

[severinejag]

Hi Kevin,

Thanks, that fixed the playback issues.

Alright, so that was another thing I had tried doing based on what you were saying earlier about importing raven tables but encountered difficulties. When do File-->Import/Export-->"Import from Raven" I get the error:

Reference to non-existent field 'squeakfolder'.

Error in import_raven_Callback (line 6) [ravenname,ravenpath] = uigetfile([handles.squeakfolder '/*.txt'],'Select Raven Log');

Error in gui_mainfcn (line 95) feval(varargin{:});

Error in DeepSqueak (line 30) gui_mainfcn(gui_State, varargin{:});

Error in matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)DeepSqueak('import_raven_Callback',hObject,eventdata,guidata(hObject)) Error while evaluating Menu Callback.

I know this is from the new function you made to support zebra vocalizations, but since you said you had experienced no issues importing, I had thought maybe there was user error.

I apologize for the additional complications.

Severine

[MxMarx]

That error sounds like it's caused by an older version. Is it possible that you have multiple copies of the function?

In the current version, line 6 of import_raven_Callback.m should be this, which is different from what the error says:

[ravenname,ravenpath] = uigetfile([handles.data.squeakfolder '/*.txt'],'Select Raven Log');

[severinejag]

I deleted all of the files I have and tried re-installing, and I think that was the issue. I have no problems now with importing Raven files or with making new training images. You can close the issue now. Thank you so much for all of your assistance.

Severine

[appozz]

Hey Severine,

You need to change your display range (Tools -> Change Display Range). Set it to 0 - 10 kHz.

Then you need to change playback rate (Tools -> Change Playback Rate).

To create the training data you need to import the raven files (File -> Import From Raven) Then choose the raven log, and corresponding audio files. The resulting .mat files will be placed in your detections folder. Then use those .mat files to create training images (Tools -> Network Training -> Create Detection Network Training Images). Choose 1 .mat at a time so you can create "bout" images.

Use bouts (.5s worked for us)

Use augmented duplicates ( 1 or 2)

Spectrogram settings for creating training images: (These were a guess on my part to make nice images and could be modified, whatever settings are normal for zebra spectrograms).

windowsize = 0.01 noverlap = 0.005 nfft = 0.02

You will then be prompted to choose the corresponding audio file. The images and training table will be generated and placed in a "Training" folder in the main DeepSqueak Directory. Repeat this for all of your .mat files.

Then you can train the network with the corresponding image tables (Tools -> Network Training -> Train Detection Network).

Sorry for how complex this is, we really wanted to maintain fleaxability when training new networks, so the GUI has a bit of a learning curve.

-Kevin

Hello Severine and Kevin, Similar to Severine's zebras, I am attempting to use DeepSqueak for a new species (parrot vocalizations; 500-12,000 Hz). I followed along with this issue earlier this year. I am new to MatLab and Raven Pro and I'm struggling to import from Raven, and to me it's still a little unclear on the way you broke down the import steps. When I go to the Raven Log folder it is empty, so after creating selections in Raven I manually place my .WAV files and .txt tables into the folder. In DeepSqueak when I select "import from Raven" I navigate to the Raven Log folder where I am first prompted to select a .txt file, and after making the selection the folder pops up again for me to choose the corresponding .WAV file. I have only had success doing this one time with one table/audio file combination, one in which I had a longer clip with multiple vocalizations, each of which was selected in Raven Pro and included in the .txt table. When I try to import my table/audio combinations with individual vocalizations (similar in duration, amplitude, and frequency to those in Severine's Training Clips) DeepSqueak is giving me errors:

`Warning: Variable names were modified to make them valid MATLAB identifiers. The original names are saved in the VariableDescriptions property. Error using audioread (line 135) Range requested is greater than the total number of samples in the file. TotalSamples is 5626.

Error in mergeAudio (line 9) audio = audioread(fname, window);

Error in import_raven_Callback (line 42) Audio(i) = {mergeAudio([audiopath audioname], audioSamples(i,:))};

Error in gui_mainfcn (line 95) feval(varargin{:});

Error in DeepSqueak (line 30) gui_mainfcn(gui_State, varargin{:});

Error in matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)DeepSqueak('import_raven_Callback',hObject,eventdata,guidata(hObject))

Error while evaluating Menu Callback.`

I appreciate your time and effort in explaining this to me, let me know if I should open my own issue