Vectorial representation, particular of multiscale STFT

[turian]

With the rise of fast vector databases for doing approximate nearest neighbors (FLANN, annoy, chroma, milvius, weaviate, etc.), it becomes increasingly useful to have vectorial representations of audio for exploring large corpora.

For something like l1 distance over log-mels, e.g. in hifigan, it is relatively straightforward to get the vector representation of audio without the need for a library like auraloss: https://github.com/jik876/hifi-gan/blob/master/meldataset.py

However, for doing l2 distance of a multiscale STFT representation, it can be relatively tricky to make sure that all different STFT scales are correctly normalized and the code is correct. For that reason, it would be amazing to have an interface for retrieving the vectorial multiscale STFT representation of audio that can be easily and directly used in a vector database.

[csteinmetz1]

Totally. So you would like an interface to pass an audio file (or batch) and return the multi-res STFT representation?

[turian]

Correct!