didi / athena

A release version for https://github.com/athena-team/athena
Apache License 2.0
124 stars 35 forks source link

load mpc model error #37

Open l2009312042 opened 3 years ago

l2009312042 commented 3 years ago

i follow the step with asr/aishell run.sh modify as hkust ,the finetune stage when load pretrain model mpc,it report the following error , the tensorflow verison is 2.3.1 cuda 10.1 cudnn7 ,is somebody met this kind of case, how to solve it ?thanks [1]<stderr>:INFO:absl:trying to restore from : examples/asr/aishell/ckpts/mpc [0]<stderr>:INFO:absl:trying to restore from : examples/asr/aishell/ckpts/mpc [0]<stderr>:INFO:absl:Loading data from examples/asr/aishell/data/dev.csv [1]<stderr>:INFO:absl:Loading data from examples/asr/aishell/data/dev.csv [0]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter [0]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter [0]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 [0]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 [0]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 [0]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 [0]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay [0]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay [0]<stderr>:WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. [0]<stderr>:WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. [0]<stderr>:INFO:absl:loading from pretrained mpc model [0]<stderr>:INFO:absl:Loading data from examples/asr/aishell/data/train.csv [1]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter [1]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter [1]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 [1]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 [1]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 [1]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 [1]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay [1]<stderr>:WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay [1]<stderr>:WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. [1]<stderr>:WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. [1]<stderr>:INFO:absl:loading from pretrained mpc model [1]<stderr>:INFO:absl:Loading data from examples/asr/aishell/data/train.csv

l2009312042 commented 3 years ago

it likely no impact on training

Some-random commented 3 years ago

Please post the whole error log along with the command and json you used.