search

chavinlo / musicgen_trainer

simple trainer for musicgen/audiocraft
GNU Affero General Public License v3.0
324 stars 37 forks source link

Plans for Encodec fine-tuning #1

Open 0xprincess opened 1 year ago

0xprincess commented 1 year ago

Hi! Do you have any plans for writing encodec fine-tuning code? The current checkpoint seems to perform really poorly on most sorts of electronic music so it's definitely a barrier for achieving high-quality generations.

rkfg commented 1 year ago

Training LoRAs would be great! That way you can expand the Encodec's capabilities to infinity by automatically loading the appropriate LoRA for the genre.

chavinlo commented 1 year ago

Hi! Do you have any plans for writing encodec fine-tuning code? The current checkpoint seems to perform really poorly on most sorts of electronic music so it's definitely a barrier for achieving high-quality generations. @0xprincess

Yes

chavinlo commented 1 year ago

Training LoRAs would be great! That way you can expand the Encodec's capabilities to infinity by automatically loading the appropriate LoRA for the genre.

will try to add LoRAs (LM, not encodec yet) for the next release

0xprincess commented 1 year ago

@chavinlo btw, I noticed that the current release of musicgen uses mono version of encodec, but the original encodec release has 48khz stereo model. If we're talking about fine-tuning, do you think that swapping the current 32khz musicgen's encodec for 48khz model is possible? I personally think that having stereo audio would be great.