GrandaddyShmax / audiocraft_plus

Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.
MIT License
561 stars 63 forks source link

Multiband diffusion not working. #44

Open Drommer-Kille opened 1 year ago

Drommer-Kille commented 1 year ago

After git pull to latest versio, the multiband diffusion does not work. I lanch AudioCraft Plus, write TEST to prompt and hit generate. It takes quite some time to generate 10sec test. Then i go to settings and change decoder to multiband_diffusion and hit generate and get long list of errors that end to this line: TypeError: issubclass() arg 1 must be a class Files that give errors: \Python\Python310\lib\site-packages\omegaconf\dictconfig.py \Python\Python310\lib\site-packages\omegaconf_utils.py Python\Python310\lib\site-packages\omegaconf\base.py Python\Python310\lib\site-packages\omegaconf\dictconfig.py

Also, latest version seem quite slow. How can i de-bug that? It is using GPU as GPU goes 100% while generating. I have RTX3090 with 24gb VRAM.

GrandaddyShmax commented 1 year ago

Sadly I myself cannot play much with multiband as I do not owe a good enough graphics card, so my only way of testing is with the free version of google colab. Can you explain more on your issue? what duration did you try to generate? and what model? Ill try to replicate the issue

Drommer-Kille commented 1 year ago

It happens even with 1sec duration and model is Large. But it seems that the problem is not in your end. I also git pulled the original repo and it gives the same error if multiband is selected. Lesson learned: Never update if it works.

Edit. Tried also with Small model and same error happens if multiband is selected. `loading MBD Traceback (most recent call last): File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\omegaconf\dictconfig.py", line 303, in getattr return self._get_impl(key=key, default_value=DEFAULT_VALUE_MARKER) File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\omegaconf\dictconfig.py", line 361, in _get_impl node = self._get_node(key=key) File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\omegaconf\dictconfig.py", line 375, in _get_node key = self._validate_and_normalize_key(key) File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\omegaconf\dictconfig.py", line 222, in _validate_and_normalize_key return self._s_validate_and_normalize_key(self._metadata.key_type, key) File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\omegaconf\dictconfig.py", line 241, in _s_validate_and_normalize_key elif issubclass(key_type, Enum): TypeError: issubclass() arg 1 must be a class

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\queueing.py", line 388, in call_prediction output = await route_utils.call_process_api( File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\route_utils.py", line 219, in call_process_api output = await app.get_blocks().process_api( File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1437, in process_api result = await self.call_function( File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1109, in call_function prediction = await anyio.to_thread.run_sync( File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, args) File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\utils.py", line 641, in wrapper response = f(args, **kwargs) File "G:\AI\audiocraft-plus\audiocraft_plus\app.py", line 844, in predict_full load_diffusion() File "G:\AI\audiocraft-plus\audiocraft_plus\app.py", line 464, in load_diffusion MBD = MultiBandDiffusion.get_mbd_musicgen() File "G:\AI\audiocraft-plus\audiocraft_plus\audiocraft\models\multibanddiffusion.py", line 74, in get_mbd_musicgen models, processors, cfgs = load_diffusion_models(path, device=device) File "G:\AI\audiocraft-plus\audiocraft_plus\audiocraft\models\loaders.py", line 130, in load_diffusion_models model = builders.get_diffusion_model(cfg) File "G:\AI\audiocraft-plus\audiocraft_plus\audiocraft\models\builders.py", line 213, in get_diffusion_model channels = cfg.channels File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\omegaconf\dictconfig.py", line 305, in getattr self._format_and_raise(key=key, value=None, cause=e) File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\omegaconf\base.py", line 95, in _format_and_raise format_and_raise( File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\omegaconf_utils.py", line 644, in format_and_raise child_node = node._get_node(key, validate_access=False) File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\omegaconf\dictconfig.py", line 375, in _get_node key = self._validate_and_normalize_key(key) File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\omegaconf\dictconfig.py", line 222, in _validate_and_normalize_key return self._s_validate_and_normalize_key(self._metadata.key_type, key) File "C:\Users\Tester\AppData\Local\Programs\Python\Python310\lib\site-packages\omegaconf\dictconfig.py", line 241, in _s_validate_and_normalize_key elif issubclass(key_type, Enum): TypeError: issubclass() arg 1 must be a class `

GrandaddyShmax commented 1 year ago

Did it work for you on previous version? like v2.0.0?

Drommer-Kille commented 1 year ago

Not sure what version, the version from say 6 weeks ago. I generated a lot with multiband, also with original repo as it generated both versions (=great for creating true stereo files).