davidsandberg / facenet

Face recognition using Tensorflow
MIT License
13.72k stars 4.8k forks source link

Error while trying to train vggface2 #982

Open cantonioupao opened 5 years ago

cantonioupao commented 5 years ago

Hi, I am fairly new to this, i was hoping you could help me with an error I encounter whenever I try to run train_softmax.py as instructed in the wiki.

here is a copy of the whole process:

D:\Daniele Appetito\facenet-master> python src/train_softmax.py --logs_base_dir ~/logs/facenet/ --models_base_dir ~/models/facenet/ --data_dir ~/data/images/output_dir/ - -image_size 160 --model_def models.inception_resnet_v1 --optimizer ADAM --learning_rate -1 --max_nrof_epochs 500 --batch_size 90 --keep_probability 0.4 --random_flip --us e_fixed_image_standardization --learning_rate_schedule_file data/learning_rate_schedule_classifier_vggface2.txt --weight_decay 5e-4 --embedding_size 512 --lfw_distance_me tric 1 --lfw_use_flipped_images --lfw_subtract_mean --validation_set_split_ratio 0.01 --validate_every_n_epochs 5 WARNING:tensorflow:From D:\Christos Antoniou\lib\site-packages\tensorflow\contrib\learn\python\learn\datasets\base.py:198: retry (from tensorflow.contrib.learn.python.lea rn.datasets.base) is deprecated and will be removed in a future version. Instructions for updating: Use the retry module or similar alternatives. Model directory: C:\Users\Ioannis/models/facenet/20190225-183551 Log directory: C:\Users\Ioannis/logs/facenet/20190225-183551 Number of classes in training set: 6 Number of examples in training set: 65 Number of classes in validation set: 6 Number of examples in validation set: 6 Building training graph 2019-02-25 18:36:16.474075: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was n ot compiled to use: AVX2 Running training 2019-02-25 18:36:30.818895: W T:\src\github\tensorflow\tensorflow\core\framework\op_kernel.cc:1273] OP_REQUIRES failed at whole_file_read_ops.cc:114 : Unknown: NewRandomA ccessFile failed to Create/Open: D:/Daniele Appetito/facenet-master/src\generative : Access is denied. ; Input/output error Traceback (most recent call last): File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\client\session.py", line 1327, in _do_call return fn(*args) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\client\session.py", line 1312, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\client\session.py", line 1420, in _call_tf_sessionrun status, run_metadata) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.OutOfRangeError: FIFOQueue '_2_batch_join/fifo_queue' is closed and has insufficient elements (requested 90, current size 0) [[Node: batch_join = QueueDequeueUpToV2[component_types=[DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch_join/f ifo_queue, _arg_batch_size_0_0)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "src/train_softmax.py", line 580, in main(parse_arguments(sys.argv[1:])) File "src/train_softmax.py", line 234, in main prelogits, prelogits_center_loss, args.random_rotate, args.random_crop, args.random_flip, prelogits_norm, args.prelogits_hist_max, args.use_fixed_image_standardizatio n) File "src/trainsoftmax.py", line 329, in train loss, , step, reglosses, prelogits_, cross_entropymean, lr_, prelogitsnorm, accuracy_, centerloss, summary_str = sess.run(tensor_list + [summaryop], feed dict=feed_dict) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\client\session.py", line 905, in run run_metadata_ptr) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\client\session.py", line 1140, in _run feed_dict_tensor, options, run_metadata) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\client\session.py", line 1321, in _do_run run_metadata) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\client\session.py", line 1340, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.OutOfRangeError: FIFOQueue '_2_batch_join/fifo_queue' is closed and has insufficient elements (requested 90, current size 0) [[Node: batch_join = QueueDequeueUpToV2[component_types=[DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch_join/f ifo_queue, _arg_batch_size_0_0)]]

Caused by op 'batch_join', defined at: File "src/train_softmax.py", line 580, in main(parse_arguments(sys.argv[1:])) File "src/train_softmax.py", line 128, in main image_batch, label_batch = facenet.create_input_pipeline(input_queue, image_size, nrof_preprocess_threads, batch_size_placeholder) File "D:\Daniele Appetito\facenet-master\src\facenet.py", line 135, in create_input_pipeline allow_smaller_final_batch=True) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\training\input.py", line 1148, in batch_join name=name) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\training\input.py", line 800, in _batch_join dequeued = queue.dequeue_up_to(batch_size, name=name) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\ops\data_flow_ops.py", line 527, in dequeue_up_to self._queue_ref, n=n, component_types=self._dtypes, name=name) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\ops\gen_data_flow_ops.py", line 3995, in queue_dequeue_up_to_v2 component_types=component_types, timeout_ms=timeout_ms, name=name) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\framework\ops.py", line 3290, in create_op op_def=op_def) File "D:\Christos Antoniou\lib\site-packages\tensorflow\python\framework\ops.py", line 1654, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

OutOfRangeError (see above for traceback): FIFOQueue '_2_batch_join/fifo_queue' is closed and has insufficient elements (requested 90, current size 0) [[Node: batch_join = QueueDequeueUpToV2[component_types=[DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch_join/f ifo_queue, _arg_batch_size_0_0)]]

I have tensorflow and facenet in the D: , and not in C:. Could that be a problem? Any help is appreciated, thanks.

AfaqSaeed commented 2 years ago

Did you find any solutions to this problem if so please tell