zlzeng / DeepFloorplan

GNU General Public License v3.0
270 stars 103 forks source link

A bug when retraining #6

Closed shgidi closed 4 years ago

shgidi commented 4 years ago

Hi, when running python main.py --phase=train I get the following error:

start training
max_step = 40000, max_ep = 223, num_batch = 179
Start Training!
Traceback (most recent call last):
  File "main.py", line 310, in <module>
    main(FLAGS)
  File "main.py", line 300, in main
    model.train(loader_dict, num_batch)
  File "main.py", line 144, in train
    [loss_value, update_value] = sess.run([loss, optim])    
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 877, in run
    run_metadata_ptr)
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1100, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1272, in _do_run
    run_metadata)
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1291, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 1, current size 0)
     [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_UINT8], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
     [[Node: shuffle_batch/_31 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_173_shuffle_batch", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

Caused by op u'shuffle_batch', defined at:
  File "main.py", line 310, in <module>
    main(FLAGS)
  File "main.py", line 296, in main
    loader_dict, num_batch = data_loader_bd_rm_from_tfrecord(batch_size=1)
  File "/home/jupyter/cloned/DeepFloorplan/net.py", line 25, in data_loader_bd_rm_from_tfrecord
    loader_dict = read_bd_rm_record('../dataset/r3d.tfrecords', batch_size=batch_size, size=512)
  File "./utils/tf_record.py", line 354, in read_bd_rm_record
    batch_size=batch_size, capacity=batch_size*256, num_threads=1, min_after_dequeue=batch_size*32)
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 1300, in shuffle_batch
    name=name)
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 846, in _shuffle_batch
    dequeued = queue.dequeue_many(batch_size, name=name)
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 476, in dequeue_many
    self._queue_ref, n=n, component_types=self._dtypes, name=name)
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3480, in queue_dequeue_many_v2
    component_types=component_types, timeout_ms=timeout_ms, name=name)
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
    return func(*args, **kwargs)
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
    op_def=op_def)
  File "/home/jupyter/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__
    self._traceback = tf_stack.extract_stack()

OutOfRangeError (see above for traceback): RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 1, current size 0)
     [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_UINT8], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
     [[Node: shuffle_batch/_31 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_173_shuffle_batch", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

My installments are as defined in to README.md. What can be the problem?