DetectionTeamUCAS / FPN_Tensorflow

This is a tensorflow re-implementation of Feature Pyramid Networks for Object Detection.
https://github.com/DetectionTeamUCAS/FPN_Tensorflow
MIT License
347 stars 132 forks source link

is_training is not found when restore ckpt from mult_gpu_train #138

Open XeChen opened 4 years ago

XeChen commented 4 years ago

When I inference my data with the checkpoint obtained from the multi_gpu_train.py, it get an error as follow:

Traceback (most recent call last): File "/root/anaconda3/envs/tensorflow12/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call return fn(*args) File "/root/anaconda3/envs/tensorflow12/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "/root/anaconda3/envs/tensorflow12/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.NotFoundError: Key is_training not found in checkpoint [[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]] [[{{node save/RestoreV2/_395}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_397_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

I can successfully inference the data with the checkpoints from train.py but it doesn't work when use the checkpoint from the multi_gpu_train.py because of the variable is_training. Please tell me why it happens. Thank you !