Closed Beinabih closed 1 year ago
um... no...
The codebook is obtained via the preprocess_audio
that takes both the model and audio waveform as inputs. It then encodes them with encodec (here called compression_model) and then returns them.
Here is where we call that function with model and audio waveform (tensor) as arguments and we get the codebook in return https://github.com/chavinlo/musicgen_trainer/blob/5fd56f0a34f8b3ee7d5249e28a75b7e92349323f/train.py#L139
Note that the codebook here is just one batch, we then concate it to match the shape of the condition_tensors.
Feel free to correct me if I'm wrong, because the trainer does not work for anything outside overfitting at the moment.
Ah sry for the confusion, your totally right...
I moved the model.compression_model.encode(wav)
out of the
process_audio
function since i am using the Audiocraft Dataloader. I think i hallucinated some of my code in your trainer.py
Hi,
Thank you for your code. It has been very helpful to me in writing my own trainer.
I think the line
codes = torch.cat([audio, audio], dim=0)
. (here) should becodes = torch.cat([codes, codes], dim=0)
otherwise you wont use your encoded codebooks, right? :)kind regards, Jonas