Run the "generate_dcase_task4_2021.py"

[YihWenWang]

Hello,

We get some errors as below when running the generate_dcase_task4_2021.py. I don't have any idea about this problem. Would you tell me what to do? Thank you!

Traceback (most recent call last): File "generate_dcase_task4_2021.py", line 794, in generate_soundscapes( File "generate_dcase_task4_2021.py", line 571, in generate_soundscapes events = instantiate_soundscape( File "generate_dcase_task4_2021.py", line 422, in instantiate_soundscape sc = add_bg(sc) File "generate_dcase_task4_2021.py", line 343, in add_bg bg_instance = sc._instantiate()["annotations"][0]["data"][0][2] File "/home/bee/anaconda3/envs/dcase21_task4/lib/python3.8/site-packages/scaper/core.py", line 1712, in _instantiate value = self._instantiate_event( File "/home/bee/anaconda3/envs/dcase21_task4/lib/python3.8/site-packages/scaper/core.py", line 1426, in _instantiate_event label = _get_value_from_dist(label_tuple, self.random_state) File "/home/bee/anaconda3/envs/dcase21_task4/lib/python3.8/site-packages/scaper/core.py", line 424, in _get_value_from_dist return SUPPORTED_DIST[dist_tuple[0]](*dist_tuple[1:], random_state=random_state) File "/home/bee/anaconda3/envs/dcase21_task4/lib/python3.8/site-packages/scaper/util.py", line 270, in _sample_choose index = random_state.randint(len(new_list_of_options)) File "mtrand.pyx", line 746, in numpy.random.mtrand.RandomState.randint File "_bounded_integers.pyx", line 1254, in numpy.random._bounded_integers._rand_int64 ValueError: low >= high

[turpaultn]

Hello, It seems to me that one of the background data folder is empty. That could throw this error (it could be good to try and get a more explicit error if that's the case). Could you please check the folder data/dcase2021/soundbank/audio/train/soundbank/background (then check the "validation" and "evaluation" by replacing "train" in the previous path). If you don't have background files there, then can you please check at the following path: data/raw_datasets/desed_soundbank/audio/train/soundbank/background (and same, check validation and evaluation).

[YihWenWang]

I found the background data folder is empty, so I downloaded it again. When I downloaded the SINS datasets (DCASE2018-task5-dev.audio), it has an error about a "..zip' 'temp file as below.

Traceback (most recent call last): File "generate_dcase_task4_2021.py", line 724, in desed.download_desed_soundbank(desed_soundbank_folder) File "/home/bee/anaconda3/envs/dcase21_task4/lib/python3.8/site-packages/desed/download.py", line 756, in download_desed_soundbank get_backgrounds_train( File "/home/bee/anaconda3/envs/dcase21_task4/lib/python3.8/site-packages/desed/download.py", line 576, in get_backgrounds_train sins_path = download_sins(destination_folder) File "/home/bee/anaconda3/envs/dcase21_task4/lib/python3.8/site-packages/desed/download.py", line 460, in download_sins download_and_unpack_archive( File "/home/bee/anaconda3/envs/dcase21_task4/lib/python3.8/site-packages/desed/download.py", line 434, in download_and_unpack_archive shutil.unpack_archive(path_dl_tar, destination_folder, format=archive_format) File "/home/bee/anaconda3/envs/dcase21_task4/lib/python3.8/shutil.py", line 1218, in unpack_archive func(filename, extract_dir, **dict(format_info[2])) File "/home/bee/anaconda3/envs/dcase21_task4/lib/python3.8/shutil.py", line 1122, in _unpack_zipfile raise ReadError("%s is not a zip file" % filename) shutil.ReadError: /data/DESED_task/recipes/dcase2021_task4_baseline/tmp_ltmdwe60/tmpnj_dd0fe..zip is not a zip file

Could you tell me how to solve? Thank you!

[turpaultn]

Ok, it seems you're not the only one to have this error, someone else reported the same error. However, I can't reproduce it here, could it come form internet connection or country restrictions ? The way to solve it is to try and download "by hand" all the zip files in this zenodo folder : https://zenodo.org/record/1247102/ Then do this:

import desed
desed.download.filter_sins("PATH/TO/DCASE2018-task5-dev",  destination_folder=os.path.join("PATH/TO/background", "sins"),
    classes_kept=["other"],
    rm_original_sins=True)

Please tell me how it goes.

[YihWenWang]

Thank you very much! Following your steps, I have solved it!

[YihWenWang]

Hello, I had trained the baseline. The multiple evaluations provided by the official dcase2021, do they come from the prediction of the student's model or teacher's model?

[turpaultn]

Hi, please this question doesn't belong to this issue anymore so let's discuss it on Slack for example. I'm still giving the answer here, but closing the issue then after. The results are coming from the predictions of the teacher model.

PS: there is a small bug about the scaling for the baseline, so we'll change it soon, more info coming on Slack.

[YihWenWang]

Hello, I understand, thank you very much. You have worked hard.

turpaultn @.***>於 2021年3月27日 週六，上午12:04寫道：

Hi, please this question doesn't belong to this issue anymore so let's discuss it on Slack for example. I'm still giving the answer here, but closing the issue then after. The results are coming from the predictions of the teacher model.

PS: there is a small bug about the scaling for the baseline, so we'll change it soon, more info coming on Slack.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DCASE-REPO/DESED_task/issues/11#issuecomment-808334568, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEV5X3XJ6XPUZQENVX4MCA3TFSWANANCNFSM4ZSG5ZGA .

[SaoYear]

Ok, it seems you're not the only one to have this error, someone else reported the same error. However, I can't reproduce it here, could it come form internet connection or country restrictions ? The way to solve it is to try and download "by hand" all the zip files in this zenodo folder : https://zenodo.org/record/1247102/ Then do this:

import desed
desed.download.filter_sins("PATH/TO/DCASE2018-task5-dev",  destination_folder=os.path.join("PATH/TO/background", "sins"),
    classes_kept=["other"],
    rm_original_sins=True)

Please tell me how it goes.

Just to explain more: If you have done the steps above, you need to re-split the dataset by desed.download.split_desed_soundbank_train_val(desed_soundbank_folder) which means you have to also regenerate your dcase2021 dataset.

A simple way of doing this is to delete the folder: YOUR\PATH\TO\DCASE\dcase2021\soundbank. If you cannot find the dcase2021 folder in raw_datasets directory-level, then you can just continue or rerun the generate.py.

More tips: Due to the quality of the internet, we may cannot download all files successfully from the given script. But we can find all links inside the DESED library, which are all included in download.py.