search

lucascesarfd / onc_dataset

This repository contains the pipeline to obtain and process ONC data from hydrophones. A compiled version of the dataset can be found at https://ieee-dataport.org/9778
MIT License
6 stars 2 forks source link

Error when downloading raw WAV data #3

Closed angus-xwu closed 1 month ago

angus-xwu commented 7 months ago

Hi Lucas

Thank you for your work on this project.

I was trying to get access to the database as you mentioned in the README file, t but it complains as follow when I trying to download the raw .WAV files

image

Can you help me to resolve this? or if you can give me some hint to fix it?

Many thanks, Angus

psmskelton commented 7 months ago

Is this repeatable? What seems to be happening is ONC is telling us there is a deployment for the hydrophone, but when we query ONC again for the files associated with that deployment, there are no files. I suspect it is caused by the .WAV/.FLAC problem I outlined in https://github.com/lucascesarfd/underwater_snd/issues/3#issuecomment-1886592048.

You can try changing lines 131-135 from:

    available_files = [
        files
        for new_filter in filters
        for files in onc_api.getListByDevice(new_filter, allPages=True)["files"]
    ]

To something like (completely untested, not on a dev machine to test it):

available_files = []
for new_filter in filters:
    print(new_filter)
    available_files.extend(onc_api.getListByDevice(new_filter, allPages=True)["files"])

This will print the ONC filter that is being sent to getListByDevice().

Edit 2024011601: Added missing empty list.

lucascesarfd commented 1 month ago

This issue was solved on the new commit. A try except was included to deal with that.