Files don't appear in "recording" menu

[wiiareonfire]

Tried following the tutorial with the provided testing data files in songexplorer/bin/songexplorer/test/data, but was unable to open any files in the "recording" dropdown (section "Manually Annotating" in the readme). The .csv file produced in the previous steps matched what was provided in the tutorial. In the GUI, the "recording" list element flashes orange, but clicking it reveals no filenames.

The relevant files in my folder at this step are

PS_..._ch3.wav  // original file from the test data
PS_..._ch3-detect.log // both generated from the previous "detect" step
PS_..._ch3-detected.csv

I replicated this issue on both Windows 11, and MacOS 12.5 (M2 Macbook Air). Below is what appears to be some relevant output from the terminal (from MacOS):

2024-05-22 19:23:06,088 Traceback (most recent call last):
  File "/Users/me/Documents/se/songexplorer-macos/songexplorer/lib/python3.10/site-packages/tornado/gen.py", line 530, in callback
    result_list.append(f.result())
  File "/Users/me/Documents/se/songexplorer-macos/songexplorer/lib/python3.10/site-packages/bokeh/server/session.py", line 95, in _needs_document_lock_wrapper
    result = func(self, *args, **kwargs)
  File "/Users/me/Documents/se/songexplorer-macos/songexplorer/lib/python3.10/site-packages/bokeh/server/session.py", line 229, in with_document_locked
    return func(*args, **kwargs)
  File "/Users/me/Documents/se/songexplorer-macos/songexplorer/lib/python3.10/site-packages/bokeh/document/callbacks.py", line 450, in wrapper
    return invoke_with_curdoc(doc, invoke)
  File "/Users/me/Documents/se/songexplorer-macos/songexplorer/lib/python3.10/site-packages/bokeh/document/callbacks.py", line 408, in invoke_with_curdoc
    return f()
  File "/Users/me/Documents/se/songexplorer-macos/songexplorer/lib/python3.10/site-packages/bokeh/document/callbacks.py", line 449, in invoke
    return f(*args, **kwargs)
  File "/Users/me/Documents/se/songexplorer-macos/songexplorer/bin/songexplorer/src/gui/model.py", line 111, in save_annotations
    iused = isused(annotation)
  File "/Users/me/Documents/se/songexplorer-macos/songexplorer/bin/songexplorer/src/gui/model.py", line 84, in isused
    return np.where([x['file']==sound['file'] and x['ticks']==sound['ticks'] \
TypeError: 'NoneType' object is not iterable

Is there a step in the tutorial that I am missing? Or is there another step required for the file to be properly recognized? Is this step sensitive to the analysis parameters (frequency range, frequency smoothing, etc)?

[bjarthur]

oh wow, thank you for your valiant effort at working with outdated documentation! reading through it with fresh eyes i now see many places that i have not updated to the new user interface. will add this to my to-do list.

in the meantime, i believe your problem is that the "labels to use" and "kinds to use" text boxes are not set properly. at this stage in the tutorial, they should be "time,frequency,neither" and "detected", respectively. like this:

nominally clicking on "label sounds" will make that button green and fill in these two textboxes as i described above automatically. is that not the case? either way, let me know if this doesn't fix the recordings pull-down menu.

please don't hesitate to submit further issues. also, can you share what you're using songexplorer for? i'm curious, and also happy to help.

[wiiareonfire]

The labels and kinds to use fields are properly populated when selecting the "label sounds" function at the top of the GUI. They also appear in the .csv file generated by the "detect" step, as expected per the documentation.

I played around with it some more today on Windows, and reproduced the error message I posted earlier. I think that is related to clicking on the spectrogram window when nothing is loaded. So perhaps that is not particularly helpful. I also found some other exceptions when trying to load the csv with annotations, but I suppose that's also to be expected when no such file exists.

Since we will have to create annotations for other analysis, could you describe the expected format of the "ground truth" annotations .csv? The excerpt in the documentation has output PS_20130625111709_ch3.wav,470151,470719,annotated,mel-sine, so I assume the columns are filename,start_sample,end_sample,annotated,label, is that correct? What else might appear in the 4th column, or do the manual annotations only contain this to indicate that it is manually annotated "ground truth" data?

I'm basically doing tech support for a friend who wants to use SongExplorer to look at wing fanning in parasitoid wasps (Leptopilina). Thank you for the quick reply, and for your help with this! We hope SongExplorer can save a lot of time with having to manually annotate hours of recordings and really speed up the data analysis involved.

[bjarthur]

so just to be clear-- do you still not see anything in the recordings pull-down menu after the kinds/labels to use text boxes have been set properly? if not, post a screen shot of the entire songexplorer window. in addition to those two text boxes, you need to have the "ground truth" textbox filled in. there should be a table in the GUI of all the kinds and labels that looks like this (on the right):

the fourth column can be

• detected, meaning that a simple signal processing heuristic is used
• annotated, meaning that a human has generated them manually
• predicted, meaning that songexplorer's neural network has found them

there is also "missed", "mistaken", and "correct", but you shouldn't need those now.

if you already have manual annotations from another workflow, use "annotated" and then train a neural network with those in songexplorer. if it's not accurate enough, then follow the iterative workflow in songexplorer to fix mistakes and train a new model.

so you have audio recordings of the wasps moving their wings? i'm not familiar with this behavior. can you suggest a paper i could read?

[bjarthur]

you don't by chance have spaces in any of the file paths or file names, do you? that would likely cause a problem.

the error you're seeing could also occur if you moved the detected.csv file elsewhere. it needs to be in a sub-folder of the groundtruth folder.

[bjarthur]

your problem could also be that the "ground truth" text box is blank or not pointing to the correct directory. see the screenshot i posted above. i'm updating the documentation with this info.