LAION-AI / CLAP

Contrastive Language-Audio Pretraining
https://arxiv.org/abs/2211.06687
Creative Commons Zero v1.0 Universal
1.42k stars 137 forks source link

error with querying audio embeddings using API #130

Open Rsalganik1123 opened 1 year ago

Rsalganik1123 commented 1 year ago

Summary I am trying to extract audio features for the Clotho dataset evaluation examples. However, when I try model.get_audio_embedding_from_filelist and model.get_audio_embedding_from_data I get two different errors.

Code For Reproducing

Downloaded Clotho dataset from https://zenodo.org/record/3490684

import laion_clap
from glob import glob 
import ipdb 
from tqdm import tqdm 
import librosa

model = laion_clap.CLAP_Module(enable_fusion=False)
model.load_ckpt()

print(model)

audio_embeddings = []  
clotho_eval_path = '/clotho/development/'
audio_files = glob(f'{clotho_eval_path}*.wav')

for audio_file in tqdm(audio_files):
    try: 
        audio_data, _ = librosa.load(audio_file, sr=48000) # sample rate should be 48000
        audio_data = audio_data.reshape(1, -1) #Make it (1,T) or (N,T)
        audio_embed = model.get_audio_embedding_from_data(x = audio_data, use_tensor=False) # ERROR
    except Exception as e: 
        print("ERROR1:", e)
    try: 
        audio_embed = model.get_audio_embedding_from_filelist(x = audio_file, use_tensor=False) #ALSO ERRROR
    except Exception as e: 
        print("ERROR2:", e)

    audio_embeddings.append(audio_embed)

And the errors I am getting:

ERROR1: GET was unable to find an engine to execute this computation
ERROR2: /anaconda3/envs/MPD2/lib/python3.8/site-packages/laion_clap/hook.py:137: UserWarning: PySoundFile failed. Trying audioread instead.
  audio_waveform, _ = librosa.load(f, sr=48000)
/anaconda3/envs/MPD2/lib/python3.8/site-packages/librosa/core/audio.py:184: FutureWarning: librosa.core.audio.__audioread_load
        Deprecated as of librosa version 0.10.0.
        It will be removed in librosa version 1.0.
  y, sr_native = __audioread_load(path, offset, duration, dtype)
IsADirectoryError: [Errno 21] Is a directory: '/'

Requirement Versions librosa 0.10.0
torchlibrosa 0.1.0
transformers 4.30.0