Open OhadCohen97 opened 1 year ago
Hi, sorry for the late reply. To use the model in different dataset, you need to construct a new dataset loader and dataset class, which you can refer from SEDDataset.
Hi, thank you for your response.
Is it ok to use the SCV2_Dataset or DESED_Dataset? I see they are regular dataset classes, which are better for loading my standard audio WAV files. What is the difference from SEDDataset?
Can HTS-AT support multi-channel audio wav?
Thank you.
Hi,
SCV2 is for speech command v2 dataset, and desed is for the sound even detection dataset, and ESC is for the ESC-50 dataset. I think SED dataset for SCV2 might be the best fit from which you can change it into your own dataset.
Yes, it is possible to support multi-channel audio, but first you might need to change the first layer to map more than one channel to the deep feature--> meaning that the pretrained model is no longer workable. Another way is that you can merge multi-channel into the single-channel, or performing the classification on multi-channel and take their average results.
Hi, Thank you for getting back to me!
1) Regarding the multi-channel audio case, I have considered using Patch Embed to process each channel and then summing them up so that it can fit within the "forward_features" function. In your experience, Is there another approach that can be taken to establish a connection between the channels for better classification?
2) Which of the hyperparameters in the 'config.py' file I need to consider in order to properly fine-tuning on my dataset (different classification task)?
Thank you.
Hi,
Thank you for your excellent work!
I want to use HTS-Audio-Transformer for my custom dataset, different classification task.
Are there any instructions on how to run the model for a different dataset? From which file should I start?
Thanks