google-research / uda

Unsupervised Data Augmentation (UDA)
https://arxiv.org/abs/1904.12848
Apache License 2.0
2.17k stars 312 forks source link

Problems of loading data set #103

Open OutstanderWang opened 3 years ago

OutstanderWang commented 3 years ago

Hi there,

When I try to run cifar10 UDA experiences on GPU, I ran into problems as follows. My dataset the prepared by using the scripts/prprocess.sh. Could you please help me out of this problem? And I see the estimator seems to be for TPU, even thought I have set the use_TPU=False

Traceback (most recent call last): File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call return fn(*args) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn target_list, run_metadata) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found. (0) Invalid argument: {{function_node inference_Dataset_map_parser_121}} Key: label. Can't parse serialized Example. [[{{node ParseSingleExample/ParseSingleExample}}]] [[IteratorGetNext]] [[IteratorGetNext/_2731]] (1) Invalid argument: {{function_node inference_Dataset_map_parser_121}} Key: label. Can't parse serialized Example. [[{{node ParseSingleExample/ParseSingleExample}}]] [[IteratorGetNext]] 0 successful operations. 0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "main.py", line 633, in tf.app.run() File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/absl/app.py", line 303, in run _run_main(main, args) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/absl/app.py", line 251, in _run_main sys.exit(main(argv)) File "main.py", line 628, in main train(hparams) File "main.py", line 589, in train estimator.train(input_fn=train_input_fn, max_steps=FLAGS.train_steps) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3035, in train rendezvous.raise_errors() File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/error_handling.py", line 136, in raise_errors six.reraise(typ, value, traceback) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/six.py", line 703, in reraise raise value File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3030, in train saving_listeners=saving_listeners) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 370, in train loss = self._train_model(input_fn, hooks, saving_listeners) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1161, in _train_model return self._train_model_default(input_fn, hooks, saving_listeners) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1195, in _train_model_default saving_listeners) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/training/monitored_session.py", line 1259, in run run_metadata=run_metadata) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/training/monitored_session.py", line 1360, in run raise six.reraise(original_exc_info) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/six.py", line 703, in reraise raise value File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/training/monitored_session.py", line 1345, in run return self._sess.run(args, *kwargs) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/training/monitored_session.py", line 1418, in run run_metadata=run_metadata) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/training/monitored_session.py", line 1176, in run return self._sess.run(args, **kwargs) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 956, in run run_metadata_ptr) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1180, in _run feed_dict_tensor, options, run_metadata) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run run_metadata) File "/homes/wwangbc/wwangbc/bin/anaconda3/envs/tf_v1/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found. (0) Invalid argument: Key: label. Can't parse serialized Example. [[{{node ParseSingleExample/ParseSingleExample}}]] [[IteratorGetNext]] [[IteratorGetNext/_2731]] (1) Invalid argument: Key: label. Can't parse serialized Example. [[{{node ParseSingleExample/ParseSingleExample}}]] [[IteratorGetNext]] 0 successful operations. 0 derived errors ignored.