How to reproduce SingAug results?

[SkyAndCloud]

Hello, I want to use SingAug but there is no related instructions.

[ftshijt]

Thanks for bringing up the issue. The augmentation and cycle consistency framework can be achieved by adding related arguments to the config files. @PeterGuoRuc Could you please try to add related instructions at https://github.com/SJTMusicTeam/Muskits/blob/main/egs/TEMPLATE/svs1/README.md ? It would be great if you can also refer to some existing configurations for the argumentation and cycle loss.

[PeterGuoRuc]

Hi, we support the augmentation and cycle consistency framework in the xiaoice model. You can find the related arguments in egs//svs1/conf/tuning/train_xiaoice_noDP.yaml. (e.g.https://github.com/SJTMusicTeam/Muskits/blob/main/egs/ofuton_p_utagoe_db/svs1/conf/tuning/train_xiaoice_noDP.yaml )

The detailed hyperparameters are as follows:

• Pitch Augmentation -- pitch_aug_min: the lower bound of pitch augment sampling -- pitch_aug_max: the upper bound of pitch augment sampling (if pitch_aug_min==0 and pitch_aug_max==0, means no pitch augmentation)
• Mixup Augmentation -- use_mixup_training: whether to use the mixup augmentation -- loss_mixup_wight: the weight of the mixup augmentation loss
• Cycle Consistency Predictor Module -- use_cycle_process: whether to use cycle-consistent predictor module during training process -- cycle_type: must be 'single' -- w_svs: the weight of the original singing voice synthesis mel-spec loss -- w_predictor: the weight of the predictor module loss learned from G.T. music score information -- w_cycle: the weight of the music score information reconstruction loss -- predict_type: the music score information used in cycle-consistent training process -- predict_criterion_type: the loss function used in music score information cycle-consistent training process

If you have any questions, please feel free to contact us! More details about the configs of SingAug will be added to the README soon.

[ftshijt]

Many thanks @PeterGuoRuc ! Noted that the code for singaug is just at runnable-level (not yet officially published with carefully design of interfaces), we will be working on that in the migration to espnet.

[ftshijt]

Since there is no further issue raised up, I will close the issue. Thanks again for your interest.