FileNotFoundError in `deepfake_asvspoof_dataset.py` for `trial_metadata.txt`

[chandlerbing65nm]

Environment

• OS: Ubuntu
• Python version: 3.10

Description

While running train_and_test.py, the script fails due to a FileNotFoundError.

Issue: FileNotFoundError for trial_metadata.txt

Log:

Traceback (most recent call last):
  File "/home/man-group/chandler/Experiments/deepfake-whisper-features/train_and_test.py", line 125, in <module>
    evaluation_config_path, model_path = train_models.train_nn(
  File "/home/man-group/chandler/Experiments/deepfake-whisper-features/train_models.py", line 65, in train_nn
    data_train, data_test = get_datasets(
  File "/home/man-group/chandler/Experiments/deepfake-whisper-features/train_models.py", line 31, in get_datasets
    data_train = DetectionDataset(
  File "/home/man-group/chandler/Experiments/deepfake-whisper-features/src/datasets/detection_dataset.py", line 38, in __init__
    datasets = self._init_datasets(
  File "/home/man-group/chandler/Experiments/deepfake-whisper-features/src/datasets/detection_dataset.py", line 70, in _init_datasets
    asvspoof_dataset = DeepFakeASVSpoofDataset(asvspoof_path, subset=subset)
  File "/home/man-group/chandler/Experiments/deepfake-whisper-features/src/datasets/deepfake_asvspoof_dataset.py", line 29, in __init__
    self.samples = self.read_protocol()
  File "/home/man-group/chandler/Experiments/deepfake-whisper-features/src/datasets/deepfake_asvspoof_dataset.py", line 56, in read_protocol
    with open(Path(self.path) / self.protocol_file_name, "r") as file:
FileNotFoundError: [Errno 2] No such file or directory: '/home/man-group/chandler/Datasets/ASVspoof2021/DF_eval/keys/CM/trial_metadata.txt'

Steps to Reproduce

1. Run python train_and_test.py --asv_path /home/man-group/chandler/Datasets/ASVspoof2021/DF_eval --in_the_wild_path /home/man-group/chandler/Datasets/release_in_the_wild --config /home/man-group/chandler/Experiments/deepfake-whisper-features/mesonet_whisper_mfcc_finetuned/config.yaml --batch_size 8 --epochs 5 --train_amount 100000 --valid_amount 25000

Expected Behavior

• No FileNotFoundError and successful script execution.

Questions

1. Is trial_metadata.txt supposed to be provided or generated by the user?

The folder structure of the ASVspoof2021 dataset when extracted is like this:

ASVspoof2021
└── DF_eval
    ├── ASVspoof2021.DF.cm.eval.trl.txt
    ├── flac
    ├── *.flac
    ├── *.flac
    └── ...
    ├── LICENSE.DF.txt
    └── README.DF.txt

[piotrkawa]

Hi, at the time of the paper being written there were no exact labels for ASVSpoof2021 DF, so we based on this keys&metadata file.

Could you please try running code with these labels?