Fix docstring and add condition_tensor attribute to lm.generate()

facebookresearch / audiocraft

Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.

MIT License

20.18k stars 2.01k forks source link

Fix docstring and add condition_tensor attribute to lm.generate() #354

Closed IntelliNik closed 5 months ago

IntelliNik commented 7 months ago

Hi, first of all thank you for your amazing work here. I really enjoy working with MusicGen and the other tools of audiocraft!

I noticed an inconsistency between the docstring and the method signature of lm.generate(). The conditions attribute wasn't mentioned in the docstring, but condition_tensors was. As I would like to use the option to provide preprocessed condition_tensors to the generation method, I've added the attribute to the given method, similarly to lm.forward(). Therefore I'd like to ask wheater it is possible to include this method parameter into the method.

Kind Regards Niklas

felixkreuk commented 5 months ago

hey, thanks for the suggestion :) i do have some concerns about using generate this way. specifically, _sample_next_token assumes there are null conditions in condition_tensor for classifier-free-guidance. which means that in this case, generate assumes that condition_tensors already contain null conditions, and if not, then the code will probably crash due to dim mismatch during the forward pass (correct me if i'm wrong).

if you have an easy way to validate the existence of null conditions, let me know, otherwise i think this pr might cause other users to crash without a proper explanation, so i tend to reject at the moment. thanks again!