search

Stinky-Tofu / Stronger-yolo

🔥Improve yolo with latest paper
MIT License
3 stars 0 forks source link

Key conv52/batch_normalization/moving_mean not found in checkpoint #27

Open muellerzr opened 5 years ago

muellerzr commented 5 years ago

2019-03-09 16:54:43.055737: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key conv52/batch_normalization/moving_mean not found in checkpoint Traceback (most recent call last): File "train.py", line 197, in YoloTrain().train() File "train.py", line 116, in train self.load.restore(self.sess, ckpt_path) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1562, in restore err, "a Variable name or other graph key that is missing") tensorflow.python.framework.errors_impl.NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Key conv52/batch_normalization/moving_mean not found in checkpoint [[node load_save/save/RestoreV2 (defined at train.py:99) = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_load_save/save/Const_0_0, load_save/save/RestoreV2/tensor_names, load_save/save/RestoreV2/shape_and_slices)]]

Caused by op u'load_save/save/RestoreV2', defined at: File "train.py", line 197, in YoloTrain().train() File "train.py", line 99, in init self.load = tf.train.Saver(self.net_var) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1102, in init self.build() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1114, in build self._build(self._filename, build_save=True, build_restore=True) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1151, in _build build_save=build_save, build_restore=build_restore) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 795, in _build_internal restore_sequentially, reshape) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 406, in _AddRestoreOps restore_sequentially) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 862, in bulk_restore return io_ops.restore_v2(filename_tensor, names, slices, dtypes) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_io_ops.py", line 1466, in restore_v2 shape_and_slices=shape_and_slices, dtypes=dtypes, name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 488, in new_func return func(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 3274, in create_op op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1770, in init self._traceback = tf_stack.extract_stack()

NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Key conv52/batch_normalization/moving_mean not found in checkpoint [[node load_save/save/RestoreV2 (defined at train.py:99) = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_load_save/save/Const_0_0, load_save/save/RestoreV2/tensor_names, load_save/save/RestoreV2/shape_and_slices)]]

Stinky-Tofu commented 5 years ago

2019-03-09 16:54:43.055737: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key conv52/batch_normalization/moving_mean not found in checkpoint Traceback (most recent call last): File "train.py", line 197, in YoloTrain().train() File "train.py", line 116, in train self.load.restore(self.sess, ckpt_path) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1562, in restore err, "a Variable name or other graph key that is missing") tensorflow.python.framework.errors_impl.NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Key conv52/batch_normalization/moving_mean not found in checkpoint [[node load_save/save/RestoreV2 (defined at train.py:99) = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_load_save/save/Const_0_0, load_save/save/RestoreV2/tensor_names, load_save/save/RestoreV2/shape_and_slices)]]

Caused by op u'load_save/save/RestoreV2', defined at: File "train.py", line 197, in YoloTrain().train() File "train.py", line 99, in init self.load = tf.train.Saver(self.net_var) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1102, in init self.build() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1114, in build self._build(self._filename, build_save=True, build_restore=True) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1151, in _build build_save=build_save, build_restore=build_restore) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 795, in _build_internal restore_sequentially, reshape) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 406, in _AddRestoreOps restore_sequentially) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 862, in bulk_restore return io_ops.restore_v2(filename_tensor, names, slices, dtypes) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_io_ops.py", line 1466, in restore_v2 shape_and_slices=shape_and_slices, dtypes=dtypes, name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 488, in new_func return func(*args, kwargs) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 3274, in create_op op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1770, in init** self._traceback = tf_stack.extract_stack()

NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Key conv52/batch_normalization/moving_mean not found in checkpoint [[node load_save/save/RestoreV2 (defined at train.py:99) = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_load_save/save/Const_0_0, load_save/save/RestoreV2/tensor_names, load_save/save/RestoreV2/shape_and_slices)]]

Is there a model file under this weight folder?

muellerzr commented 5 years ago

Yes. It is the standard coco.

Stinky-Tofu commented 5 years ago

@muellerzr Standard coco? I have rename the weights, so you can't use it.

muellerzr commented 5 years ago

Oh. What weights do I need to use if I'm attempting to train an entirely new model on something non coco related then?

Stinky-Tofu commented 5 years ago

@muellerzr If you want to fine tune on VOC, you need to follow my readme.

muellerzr commented 5 years ago

I see that, but it also asks for: weights/voc_fine_tune_initial.ckpt where is that at?

Stinky-Tofu commented 5 years ago

You need to read the file carefully. Download the yolov3-608.weights firstly. cd yolov3_to_tf python3 convert_weights.py --weights_file=yolov3.weights --dara_format=NHWC --ckpt_file=./saved_model/yolov3_608_coco_pretrained.ckpt cd .. python rename.py And then, there will be a voc_fine_tune_initial.ckpt in weights/ folder.

muellerzr commented 5 years ago

My apologies! That's my issue I bet. So sorry for the confusion.

muellerzr commented 5 years ago

I downloaded the 608 model and I get a KeyError: u'LeakyRelu'

lyqlola commented 5 years ago

I downloaded the 608 model and I get a KeyError: u'LeakyRelu'

I think it's a tensorflow version problem. Please try to follow the readme and run the project under the environment as the author required, aka, use tensorflow 1.8.0