deezer / spleeter

Deezer source separation library including pretrained models.
https://research.deezer.com/projects/spleeter.html
MIT License
25.63k stars 2.81k forks source link

How can I train spleeter based on pretrained_models #761

Open hsduren opened 2 years ago

hsduren commented 2 years ago

when I run the train command like 'spleeter train --verbose -p configs/musdb_config.json -d musdb18hq' ,and the musdb_config.json content like this : { "train_csv": "configs/musdb_train.csv", "validation_csv": "configs/musdb_validation.csv", "model_dir": "pretrained_models/2stems", "mix_name": "mix", "instrument_list": ["vocals", "other"], "sample_rate":44100, "frame_length":4096, "frame_step":1024, "T":512, "F":1024, "n_channels":2, "n_chunks_per_song":40, "separation_exponent":2, "mask_extension":"zeros", "learning_rate": 1e-4, "batch_size":4, "training_cache":"cache/training", "validation_cache":"cache/validation", "train_max_steps": 200000, "throttle_secs":1800, "random_seed":3, "save_checkpoints_steps":1000, "save_summary_steps":5, "model":{ "type":"unet.unet", "params":{ "conv_activation":"ELU", "deconv_activation":"ELU" } } } I got errors : 2 root error(s) found. (0) Not found: Key batch_normalization/beta/Adam not found in checkpoint [[node save/RestoreV2 (defined at /miniforge3/lib/python3.9/site-packages/tensorflow_estimator/python/estimator/estimator.py:1497) ]] [[save/RestoreV2/_301]] (1) Not found: Key batch_normalization/beta/Adam not found in checkpoint [[node save/RestoreV2 (defined at /miniforge3/lib/python3.9/site-packages/tensorflow_estimator/python/estimator/estimator.py:1497) ]] So,is there any way to train spleeter based on pretrained_models?

Tomp0801 commented 1 year ago

I have got the same problem and can't find an answer. I can add that it works fine, when 'model_dir' is empty (training from scratch). Apparently this error comes up, when it tries to load the downloaded 2stems model.

Is there a certain tensorflow version that needs to be installed? Is it simply not possible?

hsduren commented 1 year ago

I have no idea yet

Tomp0801 commented 1 year ago

I found a solution! It's related to this issue: #408. The problem seems to be that it defaults to the AdamOptimizer, but apparently GradientDescentOptimizer was used for the pretrained model. Instead of changing the code like suggested there, I added "optimizer": "SGD" to the config file, which also worked.