sigsep / sigsep-mus-eval

museval - source separation evaluation tools for python
https://sigsep.github.io/sigsep-mus-eval/
MIT License
202 stars 36 forks source link

AttributeError: 'numpy.ndarray' object has no attribute 'targets' #86

Closed jim79 closed 2 years ago

jim79 commented 2 years ago

Hi, This may be a very naive question. I am trying to evaluate my model using museval. Stuck in this error message "AttributeError: 'numpy.ndarray' object has no attribute 'targets' " Kindly find my code and the error message below. Thank you jim

import torch
import musdb
import openunmix
from openunmix import predict
from openunmix import evaluate
import museval
import numpy as np
from IPython.display import Audio, display

device = torch.device('cuda' if torch.cuda.is_available()else 'cpu')

path = 'path_to_musdb18hq'

mus = musdb.DB(root=path,is_wav=True,subsets="test")   # test

def estimate_evaluate(track):
    print(track.name)
    print('estimating...')

    estimates = predict.separate(
        audio=torch.as_tensor(track.audio).float(), 
        rate=track.rate,
        targets=['vocals','drums'],
        residual=False,
        device=device,
        model_str_or_path='path_to_model'
    )
    print('estimation done')

    for target, estimate in estimates.items(): 
        print('evaluating...')

        # Evaluate using museval
        scores = museval.eval_mus_track(
            track.audio, estimates, output_dir='/path_to_json'
        )
    return scores

idx = 4  
track = mus.tracks[idx]
scores = estimate_evaluate(track)
print(scores)

================= error message=========================

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_4107/2344247881.py in <module>
     41 idx = 4
     42 track = mus.tracks[idx]
---> 43 scores = estimate_evaluate(track)
     44 print(scores)

/tmp/ipykernel_4107/2344247881.py in estimate_evaluate(track)
     35         # Evaluate using museval
     36         scores = museval.eval_mus_track(
---> 37             track.audio, estimates, output_dir='path_to_out_dir_for_json file'
     38         )
     39     return scores

/anaconda/envs/open-train/lib/python3.7/site-packages/museval/__init__.py in eval_mus_track(track, user_estimates, output_dir, mode, win, hop)
    207     # therefore track.targets is an OrderedDict
    208     eval_targets = []  # save the list of target names to be evaluated
--> 209     for key, target in list(track.targets.items()):
    210         try:
    211             # try to fetch the audio from the user_results of a given key

AttributeError: 'numpy.ndarray' object has no attribute 'targets'
faroit commented 2 years ago

You have to pass track not track.audio

jim79 commented 2 years ago

@faroit , Thank you for the pointer and also sorry to trouble you further.

  1. passed track instead of track.audio to museval.eval_mus_track()

Ended up with :

"ValueError: only one element tensors can be converted to Python scalars"

  1. Tried detaching the tensors and passing them as numpy arrays to museval.eval_mus_track() as below
estimates['vocals'] = estimates['vocals'].detach().cpu().numpy()[0]
estimates['drums'] = estimates['drums'].detach().cpu().numpy()[0]

But this leads to memory error MemoryError: Unable to allocate 1.05 PiB for an array with shape (2, 12163922, 12163922) and data type float32

Thank you jim

faroit commented 2 years ago

Have you compared with https://github.com/sigsep/open-unmix-pytorch/blob/master/openunmix/evaluate.py ?

You likely just have a shape issue

jim79 commented 2 years ago

@fariot, You are right; thank you for the time and patience. This did the trick.

estimates['vocals'] = estimates['vocals'].detach().cpu().numpy()[0].T
estimates['drums'] = estimates['drums'].detach().cpu().numpy()[0].T

jim