Open sevagh opened 1 year ago
Also I tested with the numpy code (i.e. same tensors with .detach().cpu().numpy()
and got the same outcome:
File "/usr/local/lib/python3.8/dist-packages/numpy/linalg/linalg.py", line 88, in _raise_linalgerror_singular
raise LinAlgError("Singular matrix")
numpy.linalg.LinAlgError: Singular matrix
I just had a clue; since I know I use auraloss.SISDRLoss() (Scale-Invariant SDR loss, https://github.com/csteinmetz1/auraloss) with success in my neural network, I tried your si_sdr_loss
function, where I don't get the same error
So my waveforms are a problem for the Scale-Variant (or default) bss metrics, but succeed with scale invariant sdr.
On the other hand, si_bss
is still a problem.
Hi, thanks for reporting. The SIR requires computation of the inverse of the covariance matrix. From the limited output your provide, it looks like there is very high correlation between the two reference signals. If that is the case, then their covariance matrix will be close to rank deficient and the system cannot be stably inverted. This is a limitation of the SIR metric itself, which is not well defined in this case.
Nevertheless, when accuracy of the metric is not of utmost importance and you only want to ensure non-failure (e.g., while training a neural network), you can use the load_diag
parameter which will add a small diagonal loading to the covariance matrix to stabilize inversion.
Thanks for the suggestion, if I use a large enough value e.g. load_diag=1e-5
, I get past this issue.
Indeed, basically in the early iterations of my neural network, there is very little "demixing" and the output is the same as the input until some demixing is learned.
Next, I run into the cost matrix having NaNs, +inf/-inf later on in the Hungarian method. I think maybe using the full BSS metrics (SDR, SIR, SAR) as the training loss is not going to be a great idea.
Maybe I'll use SDR for training loss, and SDR/SIR/SAR for validation loss.
Ok, that's good.
I can believe that the output of the networks may be very close to each other in the beginning. However, the problem arises when the targets are the same or close to each other. You may want to check that you give the target/pred in the correct order to the loss. Note that the order is reversed between the loss functions of pytorch (pred, target) and the bss_eval metrics (target, pred). In your example above, the waveform used is the same for pred in target.
For the NaN, there is another workaround implemented. You can set clamp_db=30
(for example) and the output will be clipped in [-30 dB, 30 dB]. This should solve the problems you are having with the Hungarian method.
Hello,
I have some waveforms on which the evaluation fails. The shape is
[64, 2, 44100]
(batch size 64, 2 channels, 44100 samples i.e. 1 second of music @ 44100 Hz sample rate)I've attached the tensors (saved as .pt files), and my test looks like this:
The error output is:
The waveforms are normal stuff (extracted segments from MUSDB18-HQ). I've attached
waveform.pt
ain a zip file. Can you help me figure it out? Thanks in advance. sevagh-bss-eval-error.zip