Preprocess Issue

[ValerianFourel]

Hello Thank you for the code release: I was wondering why this error occur after launching

python main.py --fn train_audio

(amuse) Apptainer> python main.py --fn train_audio
Downloading: "https://dl.fbaipublicfiles.com/deit/deit_base_distilled_patch16_384-d0272ac0.pth" to /tmp/home/.cache/torch/hub/checkpoints/deit_base_distilled_patch16_384-d0272ac0.pth
Experiment init: AMUSE, process: 2987176, running on: g155, distributed: none, device: cuda, time: Thu Jul 25 11:04:25 2024
[RAW PATHS] Data not processed, creating all data path pickle...
[RAW PATHS] (1/7) BVH pickle created.
[RAW PATHS] (2/7) WAV pickle created.
[RAW PATHS] (3/7) TextGrid pickle created.
[RAW PATHS] (4/7) JSON pickle created.
[RAW PATHS] (5/7) Emotions pickle created.
[RAW PATHS] (6/7) Semantics pickle created.
[RAW PATHS] Eng/ total bvhs:  1 / 1 , Eng/ total wavs:  0 / 0 , Eng/ total texts:  0 / 0 , Eng/ total face jsons:  0 / 0 , all raw emotions:  0 , all raw semantics:  0
[RAW PATHS] All modalities available for takes:  0  in English only rawdata.
[RAW PATHS] Dropping takes wrt BVH for missing modalities: 
[RAW PATHS] Waves:  {'16_jorge_0_3_3'}
[RAW PATHS] JSONs:  {'16_jorge_0_3_3'}
[RAW PATHS] Texts:  {'16_jorge_0_3_3'}
[RAW PATHS] Emotions:  {'16_jorge_0_3_3'}
[RAW PATHS] Semantics:  {'16_jorge_0_3_3'}
[RAW PATHS] (7/7) All English data pickle created!
Traceback (most recent call last):
  File "main.py", line 265, in <module>
    main(args)
  File "main.py", line 85, in main
    else: full_data.preprocess(verbose=True)
  File "/tmp/home/amuse/dm/dm.py", line 322, in preprocess
    all_common_takes = list(set(all_common_takes[0]).intersection(*all_common_takes)) # Drop uncommon content
IndexError: list index out of range

[kiranchhatre]

Thank you for reaching out and for your interest in the code.

The error you're encountering suggests that there may be an issue with the raw data setup. Specifically, it looks like the script is not finding the required data files in the expected directories.

From the log snippet:

[RAW PATHS] Eng/ total bvhs:  1 / 1 , Eng/ total wavs:  0 / 0 , Eng/ total texts:  0 / 0 , Eng/ total face jsons:  0 / 0 , all raw emotions:  0 , all raw semantics:  0
[RAW PATHS] All modalities available for takes:  0  in English only rawdata.

It appears that while BVH files are present, other required files such as WAVs, Texts, JSONs, Emotions, and Semantics are missing. This discrepancy is causing the script to fail with an IndexError.

Please ensure that you follow the instructions in the download section of our webpage to correctly set up your data. Specifically:

• Download and place all the necessary files in the corresponding directories as described in the setup instructions.
• If you're using custom data, make sure it follows the format used for the training data in AMUSE. The raw data setup should match the format and structure provided in our documentation to ensure compatibility.

The error IndexError: list index out of range occurs because the script expects lists to have data, but in this case, the lists are empty due to the missing files.