search

michalfaber / tensorflow_Realtime_Multi-Person_Pose_Estimation

Multi-Person Pose Estimation project for Tensorflow 2.0 with a small and fast model based on MobilenetV3
Other
216 stars 65 forks source link

failed with DataFlow is not reentrant error #2

Closed hudengjunai closed 4 years ago

hudengjunai commented 4 years ago 
Error occurred when finalizing GeneratorDataset iterator: Cancelled: Operation was cancelled
Completed epoch 3. Saving weights...
Calculating validation losses...
Validation step 0 ...
Validation losses for epoch: 3 : Loss paf 66.06363677978516, Loss heatmap 26.9287166595459, Total loss 192.95706176757812
Start processing epoch 4
2019-12-16 03:26:21.591488: W tensorflow/core/framework/op_kernel.cc:1643] Unknown: RuntimeError: This DataFlow is not reentrant!
Traceback (most recent call last):

  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/script_ops.py", line 235, in __call__
    ret = func(*args)

  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 724, in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))

  File "/apsarapangu/disk1/dengjun.hdj/pose/tensorflow_Realtime_Multi-Person_Pose_Estimation/dataset/generators.py", line 73, in f
    for i in df:

  File "/usr/local/lib/python3.6/dist-packages/tensorpack/dataflow/parallel_map.py", line 322, in __iter__
    with self._guard, _zmq_catch_error(type(self).__name__):

  File "/usr/local/lib/python3.6/dist-packages/tensorpack/dataflow/base.py", line 36, in __enter__
    raise threading.ThreadError("This DataFlow is not reentrant!")

RuntimeError: This DataFlow is not reentrant!

what's your tensorflow2.x version ?Is this s tensorpack or tensorflow bug?

my tf version is 2.1.0-dev20191106

hudengjunai commented 4 years ago

I used the tf 2.0.0 gpu version ,it worked. docker version is docker pull tensorflow/tensorflow:2.0.0-py3-jupyter