search

eurecom-asp / rawnet2-antispoofing

This repository includes the code to reproduce our paper "End-to-end anti-spoofing with RawNet2" (https://arxiv.org/abs/2011.01108) published in ICASSP '21.
MIT License
49 stars 13 forks source link

Dataset #7

Open conheaven opened 3 months ago

conheaven commented 3 months ago

How can i eval the dataset I built myself ?

GeWanying commented 3 months ago

Hi @conheaven, one way to do this is to:

  1. Organize your dataset in the same structure as ASVspoof 2019 LA. Since you want to evaluate your own dataset, paste your files under the Eval/flac/ subfolder
  2. Generate the corresponding evaluation protocol in the same manner as the evaluation protocol of ASVspoof 2019 LA, you would need to make sure the file_ID and the labels are correct.
  3. Run the main.py code for evaluation with modifying the default argument '--database_path' and '--protocols_path' to your own paths as in 1. and 2.

Another way is to write your own script, with most lines copied from main.py, write your own dataloader so it works the same way as in this repo (audio padding, file writing) but reads any arbitrary paths and protocols you specify.

conheaven commented 3 months ago

Hi @conheaven, one way to do this is to:

  1. Organize your dataset in the same structure as ASVspoof 2019 LA. Since you want to evaluate your own dataset, paste your files under the Eval/flac/ subfolder
  2. Generate the corresponding evaluation protocol in the same manner as the evaluation protocol of ASVspoof 2019 LA, you would need to make sure the file_ID and the labels are correct.
  3. Run the main.py code for evaluation with modifying the default argument '--database_path' and '--protocols_path' to your own paths as in 1. and 2.

Another way is to write your own script, with most lines copied from main.py, write your own dataloader so it works the same way as in this repo (audio padding, file writing) but reads any arbitrary paths and protocols you specify.

Thank you for your reply! I have changed the dataloader so I can generate for my dataset. Another question, what is the scores genereted as the picture?

131de43df5f7a4a4b50120db4389c38
GeWanying commented 3 months ago

Hi @conheaven, the three columns in the screenshot are: file_ID, labels, scores.

It's clear that your classifier separate bonafide and spoof well, one tiny detail is that NORMALLY the score should indicate the probability of being bonafide, so bonafide utts should have a higher score, i.e., close to 1. I think you might need to check if you use network.output[:1] as output score, while you should use network.output[:0], or reverse.