search

craffel / mir_eval

Evaluation functions for music/audio information retrieval/signal processing algorithms.
MIT License
600 stars 112 forks source link

SIR=inf #260

Closed TeaPearce closed 6 years ago

TeaPearce commented 7 years ago

This may be a basic question but I'm struggling to get sensible results from bss_eval_sources. I am using it to assess a blind source separation system of musical mix -> vocals. To use bss_evalsources, I feed in two signals, one being the original clean vocal, the other my recreated estimated vocal. I get seemingly sensible results for SDR (that increase and decrease as I'd expect), however SAR always returns an identical answer, and SIR is always inf. I suppose einterfand e_noiseare always returned as zero but I'm not sure why. Am I doing something wrong?

code:

estimated_sources.shape # recreated acapella
(8322560,)

reference_sources.shape # clean vocal
(8322560,)

sdr, sir, sar, perm = bss_eval_sources(reference_sources, estimated_sources, compute_permutation=False)

result:

sdr [ 3.75990655] 
sir [ inf] 
sar [ 3.75990655] 
perm  [0]

I got my understanding of what's going on from this

faroit commented 7 years ago

bsseval assumes linear mixtures, therefore there is no point in comparing just reference and estimate for a single source. To correctly compute the SIR you would need to pass the interfering sources as well, e.g. [vocals, accompaniment].

If you are only interested in the quality of a single source compared to the reference, you are better off using something like peaq.