Improved preprocessing and collator

[sandorfoldi]

During preprocessing, spectrograms are now saved as torch tensors The lengths of audiofiles are also saved to a csv file

The following two collators are implemented and can be selected from cfg:

• DeleteShorts: the collator that we have been using
• ZeroPad: does not delete the samples that are too short, but zero pads them instead

The audio_lengths csv file is used when collator == DeleteShorts, and this way only the long enough audiofile paths are stored in the dataset => batch size is constant

[sandorfoldi]

I tested this on my local machine, voltash and it throws no errors. Tried to submit it as a job too, so far I got no notifications

[mustass]

@panosapos how will this extraction of audio lengths work when we have a mixture of datasets? Have you discussed that?

[panosapos]

@panosapos how will this extraction of audio lengths work when we have a mixture of datasets? Have you discussed that?

You think that the solution you suggested yesterday would not work in this case?

[mustass]

@panosapos how will this extraction of audio lengths work when we have a mixture of datasets? Have you discussed that?

You think that the solution you suggested yesterday would not work in this case?

No no, it will. I just want to be reassured that you guys talk to each other :)