search

tensorflow / models

Models and examples built with TensorFlow
Other
77.16k stars 45.75k forks source link

[FasterRCNN] Checkpoint is expected to be an object-based checkpoint_Object detection. #9278

Open lacls opened 4 years ago

lacls commented 4 years ago

Prerequisites

https://github.com/tensorflow/models/tree/master/research/object_detection

1. Describe the bug

My running on the terminal

python object_detection/model_main_tf2.py \
    --pipeline_config_path=/home/MaskRCNN_DeepFashion2/models/research/object_detection/configs/tf2/faster_rcnn_resnet101_v1_800x1333_coco17_gpu-8.config \
    --model_dir=home/MaskRCNN_DeepFashion2/Mask_RCNN/CP \
    --num_train_steps=10000 \
    --sample_1_of_n_eval_examples=1 \
    --alsologtostderr

The error I got is below

  File "object_detection/model_main_tf2.py", line 113, in <module>
    tf.compat.v1.app.run()
  File "/home/anaconda3/lib/python3.7/site-packages/tensorflow/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/anaconda3/lib/python3.7/site-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/home/anaconda3/lib/python3.7/site-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "object_detection/model_main_tf2.py", line 110, in main
    record_summaries=FLAGS.record_summaries)
  File "/home/anaconda3/lib/python3.7/site-packages/object_detection-0.1-py3.7.egg/object_detection/model_lib_v2.py", line 569, in train_loop
    unpad_groundtruth_tensors)
  File "/home/anaconda3/lib/python3.7/site-packages/object_detection-0.1-py3.7.egg/object_detection/model_lib_v2.py", line 346, in load_fine_tune_checkpoint
    raise IOError('Checkpoint is expected to be an object-based checkpoint.')
OSError: Checkpoint is expected to be an object-based checkpoint.

Here is my config file with ONLY Fine-tune part

 }
  gradient_clipping_by_norm: 10.0
  fine_tune_checkpoint: "/home/MaskRCNN_DeepFashion2/pre_trained_models/faster_rcnn_resnet101_coco_2018_01_28/model.ckpt"
  fine_tune_checkpoint_type: "detection"

The folder of pre_trained_models consists of

Screen Shot 2020-09-22 at 07 12 21

Thanks you guys, again!

6. System information

bluemix commented 4 years ago

the same error here

IvanGarcia7 commented 4 years ago

Anyone know the solution? I have the same error

bluemix commented 4 years ago

since this issue is not solved yet, I am currently using: https://github.com/ultralytics/yolov5 @IvanGarcia7

IvanGarcia7 commented 4 years ago

Thanks @bluemix I will check it

S130111 commented 3 years ago

Hello, is this problem solved?

S130111 commented 3 years ago

@lacls

benballintyn commented 3 years ago

I'm having a similar issue trying to retrain ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03.

I get this after changing the fine_tune_checkpoint field in pipeline.config to point to model.ckpt (instead of the full file name as suggested here: https://github.com/tensorflow/models/issues/2676)

abark-electric commented 3 years ago

Having the same issue here. Any updates?

Abdullamhd commented 3 years ago

anybody solved the issue ?

HackBrettHB commented 3 years ago

I would also be interested in a hint, what is going wrong...

x1001000 commented 3 years ago

ckpt.index needs not to be renamed

HackBrettHB commented 3 years ago

I did not rename anything. The dir "checkpoint" includes a file named "ckpt-0.index" and in "checkpoint" it also states: model_checkpoint_path: "ckpt-0" all_model_checkpoint_paths: "ckpt-0"

After some try and error I now changed my fine_tune_checkpoint in the config to point to /checkpoint/ckpt-0 instead of /checkpoint/ckpt-0.index which now at least loads the checkpoint.

x1001000 commented 3 years ago

@HackBrettHB That's what I meant. Point to /checkpoint/ckpt-0 and not to rename /checkpoint/ckpt-0.index

floflif commented 3 years ago

I'm having a similar issue trying to retrain ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03.

I get this after changing the fine_tune_checkpoint field in pipeline.config to point to model.ckpt (instead of the full file name as suggested here: #2676)

Hello, sorry for the inconvenience but I have currently the same issue. I'm using Tensorflow 2.5.0 with the right CUDA version, my model is ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03 and so I modified the associated config file : ssd_mobilenet_v2_quantized_300x300_coco.config (from https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/ssd_mobilenet_v2_quantized_300x300_coco.config)

I did put the right path for me that is : fine_tune_checkpoint: "C:/tensorflow1/models/research/object_detection/ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03/model.ckpt" I also have my own labelmap.pbtxt, my train.record and test.record

But indeed, this is telling me the same error as the top of this topic : 

Traceback (most recent call last):
    File "model_main_tf2.py", line 115, in <module>
      tf.compat.v1.app.run()
    File "C:\Users\Flo\mob1\lib\site-packages\tensorflow\python\platform\app.py", line 40, in run
      _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
    File "C:\Users\Flo\mob1\lib\site-packages\absl\app.py", line 312, in run
      _run_main(main, args)
    File "C:\Users\Flo\mob1\lib\site-packages\absl\app.py", line 258, in _run_main
      sys.exit(main(argv))
    File "model_main_tf2.py", line 112, in main
      record_summaries=FLAGS.record_summaries)
    File "C:\tensorflow1\models\research\object_detection\model_lib_v2.py", line 603, in train_loop
      train_input, unpad_groundtruth_tensors)
    File "C:\tensorflow1\models\research\object_detection\model_lib_v2.py", line 389, in load_fine_tune_checkpoint
      raise IOError('Checkpoint is expected to be an object-based checkpoint.')
  OSError: Checkpoint is expected to be an object-based checkpoint.