search

facebookresearch / audiocraft

Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.
MIT License
20.15k stars 2.01k forks source link

Clarification on Hardware Utilization by MusicGen in AudioCraft #435

Open tebjan opened 3 months ago

tebjan commented 3 months ago

Hi AudioCraft Team,

While generating music clips with MusicGen on Windows 11, I've noticed my CPU cores are heavily utilized, whereas my NVIDIA GPU Laptop (16GB VRAM) usage is minimal (~5-10%). This is intriguing, especially when compared to applications like Stable Diffusion that predominantly utilize GPU. My laptop also has an integrated Intel GPU, could that create some conflicts?

Questions:

Is MusicGen designed to rely more on CPU than GPU? If so, what's the expected GPU usage? Optimal Settings: Any tips for ensuring the dedicated GPU is effectively used?

yukara-ikemiya commented 3 months ago

Is MusicGen designed to rely more on CPU than GPU?

If you specify to GPUs for generation, I believe the CPU usage should not be abnormally high compared to other scripts like Stable Diffusion because CPU is used only for managing GPU processes and jobs executed on CPU.

Lower usage of GPU arises from the feature of autoregressive (AR) models like MusicGen. AR models can generate samples with key-value caching technique that drastically reduce computational cost and memory consumption, which results in lower GPU usage.

I guess you can increase batch size during sampling more to effectively generate audio samples.

ayushhroyy commented 2 months ago

how do you specify gpu for generation?