search

solivr / tf-crnn

TensorFlow convolutional recurrent neural network (CRNN) for text recognition
GNU General Public License v3.0
292 stars 98 forks source link

OutOfRange Problem #26

Closed BouOus closed 6 years ago

BouOus commented 6 years ago

Hi,

mcq-2@mcq-2-linux:~/projects/tf-crnn$ python3 train.py -g 0 -ft /home/mcq-2/Documents/Dataset/DataSet_Text/Train.csv -fe /home/mcq-2/Documents/Dataset/DataSet_Text/Test.csv -o /home/mcq-2/Documents/Dataset/DataSet_Text/ /usr/local/lib/python3.5/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters 0%| | 0/6 [00:00<?, ?it/s]2018-02-27 16:00:07.305215: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA 2018-02-27 16:00:07.419715: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2018-02-27 16:00:07.419970: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties: name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.6705 pciBusID: 0000:01:00.0 totalMemory: 10.91GiB freeMemory: 10.29GiB 2018-02-27 16:00:07.419982: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1) 2018-02-27 16:00:09.022837: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_3_prepared_batch_queue/random_shuffle_queue' is closed and has insufficient elements (requested 128, current size 0) [[Node: prepared_batch_queue = QueueDequeueManyV2[component_types=[DT_STRING, DT_FLOAT, DT_INT32, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](prepared_batch_queue/random_shuffle_queue, prepared_batch_queue/n)]] 2018-02-27 16:00:09.022902: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_3_prepared_batch_queue/random_shuffle_queue' is closed and has insufficient elements (requested 128, current size 0) [[Node: prepared_batch_queue = QueueDequeueManyV2[component_types=[DT_STRING, DT_FLOAT, DT_INT32, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](prepared_batch_queue/random_shuffle_queue, prepared_batch_queue/n)]] 2018-02-27 16:00:09.023102: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_3_prepared_batch_queue/random_shuffle_queue' is closed and has insufficient elements (requested 128, current size 0) [[Node: prepared_batch_queue = QueueDequeueManyV2[component_types=[DT_STRING, DT_FLOAT, DT_INT32, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](prepared_batch_queue/random_shuffle_queue, prepared_batch_queue/n)]]

Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 783, in _trainmodel , loss = mon_sess.run([estimator_spec.train_op, estimator_spec.loss]) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/monitored_session.py", line 521, in run run_metadata=run_metadata) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/monitored_session.py", line 892, in run run_metadata=run_metadata) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/monitored_session.py", line 965, in run raise six.reraise(original_exc_info) File "/usr/local/lib/python3.5/dist-packages/six.py", line 693, in reraise raise value File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/monitored_session.py", line 952, in run return self._sess.run(args, *kwargs) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/monitored_session.py", line 1024, in run run_metadata=run_metadata) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/monitored_session.py", line 827, in run return self._sess.run(args, **kwargs) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 889, in run run_metadata_ptr) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1120, in _run feed_dict_tensor, options, run_metadata) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1317, in _do_run options, run_metadata) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1336, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_3_prepared_batch_queue/random_shuffle_queue' is closed and has insufficient elements (requested 128, current size 0) [[Node: prepared_batch_queue = QueueDequeueManyV2[component_types=[DT_STRING, DT_FLOAT, DT_INT32, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](prepared_batch_queue/random_shuffle_queue, prepared_batch_queue/n)]]

Caused by op 'prepared_batch_queue', defined at: File "train.py", line 97, in image_summaries=False)) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 302, in train loss = self._train_model(input_fn, hooks, saving_listeners) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 708, in _train_model input_fn, model_fn_lib.ModeKeys.TRAIN) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 577, in _get_features_and_labels_from_input_fn result = self._call_input_fn(input_fn, mode) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 663, in _call_input_fn return input_fn(**kwargs) File "/home/mcq-2/projects/tf-crnn/src/data_handler.py", line 37, in input_fn name='prepared_batch_queue') File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/input.py", line 1225, in shuffle_batch name=name) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/input.py", line 796, in _shuffle_batch dequeued = queue.dequeue_many(batch_size, name=name) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/data_flow_ops.py", line 464, in dequeue_many self._queue_ref, n=n, component_types=self._dtypes, name=name) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 2418, in _queue_dequeue_many_v2 component_types=component_types, timeout_ms=timeout_ms, name=name) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2956, in create_op op_def=op_def) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1470, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

OutOfRangeError (see above for traceback): RandomShuffleQueue '_3_prepared_batch_queue/random_shuffle_queue' is closed and has insufficient elements (requested 128, current size 0) [[Node: prepared_batch_queue = QueueDequeueManyV2[component_types=[DT_STRING, DT_FLOAT, DT_INT32, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](prepared_batch_queue/random_shuffle_queue, prepared_batch_queue/n)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "train.py", line 97, in image_summaries=False)) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 302, in train loss = self._train_model(input_fn, hooks, saving_listeners) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 783, in _trainmodel , loss = mon_sess.run([estimator_spec.train_op, estimator_spec.loss]) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/monitored_session.py", line 537, in exit self._close_internal(exception_type) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/monitored_session.py", line 574, in _close_internal self._sess.close() File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/monitored_session.py", line 820, in close self._sess.close() File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/monitored_session.py", line 941, in close ignore_live_threads=True) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/coordinator.py", line 389, in join six.reraise(*self._exc_info_to_raise) File "/usr/local/lib/python3.5/dist-packages/six.py", line 692, in reraise raise value.with_traceback(tb) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/queue_runner_impl.py", line 238, in _run enqueue_callable() File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1231, in _single_operation_run target_list_as_strings, status, None) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: Expect 2 fields but have 1 in record 0 [[Node: csv_reading_op = DecodeCSV[OUT_TYPE=[DT_STRING, DT_STRING], field_delim=";", na_value="", use_quote_delim=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](file_reading_op:1, csv_reading_op/record_defaults_0, csv_reading_op/record_defaults_0)]]

can you help me please ?

Thank you

solivr commented 6 years ago

Hi, Expect 2 fields but have 1 in record 0 [[Node: csv_reading_op = DecodeCSV[OUT_TYPE=[DT_STRING, DT_STRING], field_delim=";", na_value="", use_quote_delim=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](file_reading_op:1, csv_reading_op/record_defaults_0, csv_reading_op/record_defaults_0)]] looks more like a CSV decoding error. Did you follow the steps in the README.txt concerning the format of the CSV ?

BouOus commented 6 years ago

each row of my csv file contains image path + label, like this way /home/Documents/Set/1.jpg;66 the file contains : 108000 rows * 1 col

am I wrong in something ?

solivr commented 6 years ago

/home/Documents/Set/1.jpg;66 looks correct. Have you checked that there is no blank line (or any other special character) at the beginning or end of the file ?

BouOus commented 6 years ago

Thank you very much, i have solved the problem. There were comas at the beginning and at the end of each line in my csv file