Update KWS, MSNoise, Signalmixer Data Loaders & Evaluation Notebook, Add New Scripts for Mixed Signals

[EyubogluMerve]

Updates

• Updates on kws20.py:

  • Dataset functions for signalmixer are added to the KWS data loader.
  • White Noise application is updated using SNR levels (generated between [-5, 20] dB).

• Updates on msnoise.py:

  • Unnecessary reading operations are removed (filter_classes & filter_dtype)
  • A probability array is added as a parameter to MSnoise class. (Holds the noise type probabilities)
  • Naming differences in the raw Train/Test data are handled during download operation.

• Updates on signalmixer.py:

  • Several parameters are implemented: apply_prob: noise application probability (0.8, default) snr_range: SNR range to be applied to the dataset ([-5 ,10] dB SNR, default) noise_type: array that contains the noise type names (All Noise Types, default)

• Evaluation Notebook for KWS is updated.

• Scripts for KWS NAS & v3 models are added for mixing with MSnoise applications.

[EyubogluMerve]

Could you also have a look at scripts/evaluate_kws20_v3_SNR10dB.sh, and check that the MixedKWS20_10dB dataset still works, with all the changes we made to the kws20 and msnoise datasets?

Even better, I think it would be best if we integrate MixedKWS20_10dB dataset into signalmixer.py for backward compatibility purposes, then delete the unmaintained mixedkws.py. Having two files for the same purpose can be confusing.

I created the same dataset (MixedKWS20_10dB) using signalmixer.py data loader under kws20.py file. Also, I removed the mixedkws.py. In addition, I made some fixes in msnoise.py & kws20.py.

I checked all the evaluation and training scripts just in case. They're all working correctly.

[EyubogluMerve]

For kws20.py benchmark dataset option is added. According _getdataset function is added. benchmark parameter is set to False in order to get the regular kws20 dataset. Scripts for benchmark train + evaluate is added to the PR.

[alicangok]

@EyubogluMerve, could you please update your PR with the following patch I created?: MervePR.patch Thank you.

List of changes:

• class names now follow the original dataset convention (e.g. SILENCE -> _silence_)
• training and validation examples for the _silence_ class now come from all possible _background_noise_ types, instead of exclusively using running_tap for validation (as we discussed in Teams)
• the benchmark testing dataset now includes the _silence_ samples
• for the MSNoise dataset, the download parameter is now set to True
• modifications and additions to various comments and printed logs

---

Note: This PR contains the evaluate_kws12_nas_benchmark.sh script, which requires a model checkpoint for the 12-class case which currently does not exist in the ongoing PR to the synthesis repo. We will either need to remove this script from this PR, or add the trained checkpoint file to the synthesis repo. (If we decide on the latter, I will make a separate PR to the synthesis repo)

[alicangok]

@EyubogluMerve, could you apply MervePR2.patch and update the PR? Other than that, it LGTM.

As @rotx-maxim requested here, this patch introduces constants to deal with the "data_type"s and replaces the | symbol with the explicit torch.logical_or() function for the index merging operation for better readability.