Closed zhangchi2004 closed 1 month ago
Meanwhile, I wonder how we synchronize the EMA update of the codebook in multi-gpus. I haven't found any relevant parts in the code.
I see. I think you should increase your learning rate (3e-4 or 1e-3). And could you tell me why you change the ratios from [8,5,4,2] to [8,5,4,4] ? It means the encoder have 640x downsample rate and it could loss many audio informations.
If you have 50000 waveforms, maybe overfit ?
I see. I think you should increase your learning rate (3e-4 or 1e-3). And could you tell me why you change the ratios from [8,5,4,2] to [8,5,4,4] ? It means the encoder have 640x downsample rate and it could loss many audio informations.
Thank you. I have tried 3e-4 and waiting for the result. I changed the ratios in order to compare encodec with other compression methods at an equal compression rate.
I see. I think you should increase your learning rate (3e-4 or 1e-3). And could you tell me why you change the ratios from [8,5,4,2] to [8,5,4,4] ? It means the encoder have 640x downsample rate and it could loss many audio informations.
Thank you. I have tried 3e-4 and waiting for the result. I changed the ratios in order to compare encodec with other compression methods at an equal compression rate.
yeah,maybe the compression rate is so large that decrease model's performance. BTW, multi gpu codebook update you can check https://github.com/ZhikangNiu/encodec-pytorch/blob/c6b6de91c4bfeb8582b5e51f1a8b599e04b7d860/quantization/core_vq.py#L157
I see. I think you should increase your learning rate (3e-4 or 1e-3). And could you tell me why you change the ratios from [8,5,4,2] to [8,5,4,4] ? It means the encoder have 640x downsample rate and it could loss many audio informations.
Thank you. I have tried 3e-4 and waiting for the result. I changed the ratios in order to compare encodec with other compression methods at an equal compression rate.
yeah,maybe the compression rate is so large that decrease model's performance. BTW, multi gpu codebook update you can check
I have two questions regarding this. Firstly, in the original code these two lines are commented, and why doesn't it affect performance? Second, the buffers are only broadcasted in the initialization and after expiration, but during normal EMA optimization (self.ema_inplace
) I don't see any broadcast. In my understanding the codebook vectors will move differently on different gpus due to different data.
I see. I think you should increase your learning rate (3e-4 or 1e-3). And could you tell me why you change the ratios from [8,5,4,2] to [8,5,4,4] ? It means the encoder have 640x downsample rate and it could loss many audio informations.
Thank you. I have tried 3e-4 and waiting for the result. I changed the ratios in order to compare encodec with other compression methods at an equal compression rate.
yeah,maybe the compression rate is so large that decrease model's performance. BTW, multi gpu codebook update you can check https://github.com/ZhikangNiu/encodec-pytorch/blob/c6b6de91c4bfeb8582b5e51f1a8b599e04b7d860/quantization/core_vq.py#L157
I have two questions regarding this. Firstly, in the original code these two lines are commented, and why doesn't it affect performance? Second, the buffers are only broadcasted in the initialization and after expiration, but during normal EMA optimization (
self.ema_inplace
) I don't see any broadcast. In my understanding the codebook vectors will move differently on different gpus due to different data.
I attempt to train the encodec model on a 16kHz dataset with about 50000 waveforms. I am training on 8 gpus on 2 machines. I use tensor_cut = 65536, batch_size = 32(each gpu), ratios = [8,5,4,4], and lr = 5e-5. (Other configs set as default.) The model loss can converge to about the following values:
But the reconstructed audio is horrible. I would like to ask whether these loss values can properly indicate that the model can fit the dataset? Also, are there any issues with my config, and are there any other considerations for setting the config? Thank you!