I0313 15:10:55.695348 140095594268544 learning.py:507] global step 2290: loss = 24.0642 (4.784 sec/step)
INFO:tensorflow:Saving checkpoint to path ./train_model/model.ckpt
I0313 15:11:12.685011 140091937531648 supervisor.py:1117] Saving checkpoint to path ./train_model/model.ckpt
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.DataLossError'>, corrupted record at 12
[[{{node pascalvoc_2007_data_provider/parallel_read/ReaderReadV2}}]]
I0313 15:11:19.223580 140091971102464 coordinator.py:224] Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.DataLossError'>, corrupted record at 12
[[{{node pascalvoc_2007_data_provider/parallel_read/ReaderReadV2}}]]
INFO:tensorflow:Caught OutOfRangeError. Stopping Training. 2 root error(s) found.
(0) Out of range: FIFOQueue '_6_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[node fifo_queue_Dequeue (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
(1) Out of range: FIFOQueue '_6_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[node fifo_queue_Dequeue (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
[[fifo_queue_Dequeue/_321]]
0 successful operations.
0 derived errors ignored.
I0313 15:10:55.695348 140095594268544 learning.py:507] global step 2290: loss = 24.0642 (4.784 sec/step) INFO:tensorflow:Saving checkpoint to path ./train_model/model.ckpt I0313 15:11:12.685011 140091937531648 supervisor.py:1117] Saving checkpoint to path ./train_model/model.ckpt INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.DataLossError'>, corrupted record at 12 [[{{node pascalvoc_2007_data_provider/parallel_read/ReaderReadV2}}]] I0313 15:11:19.223580 140091971102464 coordinator.py:224] Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.DataLossError'>, corrupted record at 12 [[{{node pascalvoc_2007_data_provider/parallel_read/ReaderReadV2}}]] INFO:tensorflow:Caught OutOfRangeError. Stopping Training. 2 root error(s) found. (0) Out of range: FIFOQueue '_6_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0) [[node fifo_queue_Dequeue (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]] (1) Out of range: FIFOQueue '_6_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0) [[node fifo_queue_Dequeue (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]] [[fifo_queue_Dequeue/_321]] 0 successful operations. 0 derived errors ignored.
Originally posted by @haobabuhaoba in https://github.com/balancap/SSD-Tensorflow/issues/366#issuecomment-599002256