search

sigsep / sigsep-mus-eval

museval - source separation evaluation tools for python
https://sigsep.github.io/sigsep-mus-eval/
MIT License
203 stars 36 forks source link

Using BSSEval in this package without using musdb #51

Closed zingmars closed 5 years ago

zingmars commented 5 years ago

Hello,

I would like to use BSSEval to test the results of vocal separation, and found BSSEval. I have the original mixture, vocals and accompaniment (although I can get full stems for most of the songs, but that would take time on my end), and an estimated vocal and accompaniment. It would appear that all of the examples and docs assume that one is using musdb. Is it possible to use BSSEval in this package to calculate the SDR, SIR and SAR values for vocal separation without using musdb, and if yes - how would one do so?

faroit commented 5 years ago

Hi

sure, museval.metrics.bss_eval works with any numpy arrays. See documentation here.

I agree that the documentation could better mention this. Thats why we will probably move out the metrics to a separate package soon, feel free to join the discussion in #22.

zingmars commented 5 years ago

Hm, I looked at that but wasn't quite sure how to use it in my situation. Simply passing both original vocals results in https://github.com/sigsep/sigsep-mus-eval/issues/36, which makes sense I guess, so I tried adding concatenating multiple sources (original vocals, accompaniment in reference array, estimated vocals and accompaniment in estimates array), but I just get this -

Traceback (most recent call last):
  File "main.py", line 83, in <module>
    evaluator.calculate_metrics()
  File "C:\Users\zingmars\Documents\Projects\separate\evaluate.py", line 86, in calculate_metrics
    sdr, _, sir, sar, _ = museval.metrics.bss_eval(original_data, estimated_data, window=np.inf, hop=0)
  File "C:\Program Files\Python37\lib\site-packages\museval\metrics.py", line 283, in bss_eval
    (G, sf, C) = compute_GsfC()
  File "C:\Program Files\Python37\lib\site-packages\museval\metrics.py", line 259, in compute_GsfC
    reference_sources[:, win], filters_len
  File "C:\Program Files\Python37\lib\site-packages\museval\metrics.py", line 525, in _compute_reference_correlations
    sf = scipy.fftpack.fft(reference_sources, n=n_fft, axis=2)
  File "C:\Program Files\Python37\lib\site-packages\scipy\fftpack\basic.py", line 284, in fft
    return work_function(tmp,n,1,0,overwrite_x)
MemoryError

Which is odd because I'm running 64-bit python and memory usage doesn't even go above 300MiB before it crashes.

faroit commented 5 years ago

Did you pass the right shapes?

zingmars commented 5 years ago

Oh wow, I should probably avoid writing code at 2am next time. My arrays were 

(2, 1, 5248768)
(2, 1, 5248768)

instead of 

(2, 5248768, 1)
(2, 5248768, 1)

Thank you for your help!