Open haha010508 opened 3 weeks ago
some error in separation/audio_train.py 20# import look2hear.system 21# import look2hear.datas and the train code do not choose train dataset
I have updated the codebase. Please check the latest version.
Feel free to let me know if you need any further assistance!
could you share the train config file? thanks! include cfg.datas.target, cfg.model.target,cfg.optimizer.target,cfg.scheduler.target,cfg.loss.target,cfg.metrics.target,cfg.system.target, in audio_train.py file, thanks a lot!
I'll recheck the code later.
i found a bug at SonicSim/separation/train.py line: 104 to_save = system.audio_model.serialize() do not define.
i run the train code, and finish the train. but i got an error: No such file or directory: "" is mean the checkpoint.best_model_path is not set in yaml file, so what the code do? i am very confuse that.
by the way. i rewrite the best_k_models.json file, and i forget the content, if i retrain, i need a lot of time, so, could you give me an example for the best_k_models.json, thanks a lot.
trainer.fit(system, datamodule=datamodule) print_only("Training finished!") best_k = {k: v.item() for k, v in checkpoint.best_k_models.items()} with open(os.path.join(cfg.exp.dir, cfg.exp.name, "best_k_models.json"), "w") as f: json.dump(best_k, f, indent=0) state_dict = torch.load(checkpoint.best_model_path) system.load_state_dict(state_dict=state_dict["state_dict"]) system.cpu()
this is the yaml file for checkpoints
checkpoint: _target_: pytorch_lightning.callbacks.ModelCheckpoint dirpath: ${exp.dir}/${exp.name}/checkpoints monitor: val_loss mode: min verbose: true save_top_k: 5 save_last: true filename: '{epoch}-{val_loss:.4f}'
{
"Exps/bsrnn-2spk-noise-tencent/checkpoints/epoch=8-val_loss=-8.5521.ckpt": -8.552091598510742,
"Exps/bsrnn-2spk-noise-tencent/checkpoints/epoch=11-val_loss=-8.7302.ckpt": -8.730206489562988,
"Exps/bsrnn-2spk-noise-tencent/checkpoints/epoch=12-val_loss=-8.6079.ckpt": -8.6078519821167,
"Exps/bsrnn-2spk-noise-tencent/checkpoints/epoch=13-val_loss=-8.6066.ckpt": -8.606582641601562,
"Exps/bsrnn-2spk-noise-tencent/checkpoints/epoch=15-val_loss=-8.6824.ckpt": -8.682445526123047
}
could you share the train code for those model? thanks