fnlandini
commented
2 weeks ago Hi @leo67867 I guess you are choosing the epoch based on the loss on development set, correct? I am attaching the tensorboard.zip logs. You can see that in my case also the dev loss reached the lowest in just a few epochs. However, if you look at the dev_DER, you will see that it can still improve further. I observed this in a few of the fine-tuning steps with different sets. The attractor existence loss eventually grows but the BCE activation one improves on the dev set. Have you tried evaluating your model for epoch 200? I expect you will get results more similar to mine. You can also try using the whole train set (instead of 80% of it) and run for 200 epochs as I did. In this case, since I used the test set as validation, I did not choose the number of epochs very carefully and just picked 200 as a reasonable guess. My other question is if you are mixing the channels of AISHELL-4 to obtain the waveforms. That is what I did and I did not try using a single channel, for example. I am not sure if using a single channel might cause large degradation.
I hope this helps. Federico
leo67867
Hello, I am processing and training the AISHELL-4 dataset using the command:
python diaper/train.py -c DiaPer/models/10attractors/SC_LibriSpeech_2spk_adapted1-10_finetuneAISHELL4mix/train.yaml,
where I modified init_epochs: 0 and init_model_path: ''. I split the AISHELL-4 dataset into a training and validation set with an 8:2 ratio. After training, I tested my model on the test set, but the results were different from yours. For example, for L_R003S01C02, I got a DER of 72.57, whereas yours was 47.37. For M_R003S01C01, my result was 65.53, but yours was 34.28. Also, I got my best results around the 10th epoch, whereas you got yours around the 190th-200th epoch.
Could you please share the specific details of your data splitting or processing methods? Do you have any suggestions on what might be going wrong with my approach and how I can improve it? Thank you.