search

cfoster0 / CLAP

Contrastive Language-Audio Pretraining
BSD 3-Clause "New" or "Revised" License
87 stars 4 forks source link

Alternative Loss Functions #16

Open cfoster0 opened 3 years ago

cfoster0 commented 3 years ago

An alternative to the symmetric cross entropy loss, we could optimize for alignment and uniformity on the unit hypersphere as was done in this paper:

https://arxiv.org/abs/2005.10242

The code itself is like 4 lines of PyTorch, with the caveat that we'd need to write a pdist JAX function in order to use it.

cfoster0 commented 3 years ago

May be worthwhile to play with an additional speech-domain loss in the form of DINO, which should be relatively easy to implement, with a bit of JAX code and some crop augmentations.

https://arxiv.org/abs/2104.14294