tensorflow / tpu

Reference models and tools for Cloud TPUs.
https://cloud.google.com/tpu/
Apache License 2.0
5.21k stars 1.77k forks source link

Directory structure for EfficientNet training #400

Closed RubensZimbres closed 5 years ago

RubensZimbres commented 5 years ago

I'm trying to replicate EfficientNet training with ImageNet, but the directory structure is not clear for me. When I extract ImageNet from ImageNet Object Localization Challenge, I get:

ILSVRC/Annotations/CLS-LOC/train/ILSVRC2012_train_0012345.JPEG containing .xml files

ILSVRC/Data/CLS-LOC/train/ILSVRC2012_train_0012345.JPEG containing JPEG files

ILSVRC/Data/CLS-LOC/val/ILSVRC2012_val_0012345.JPEG containing JPEG files

When I try to train EfficientNet on a Google Cloud TPU, I get an error that the folder /train- was not found:

Caused by op u'input_pipeline_task0/list_files/assert_not_empty/Assert', defined at:
  File "main.py", line 725, in <module>
    app.run(main)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "main.py", line 697, in main
    est.train(input_fn=imagenet_train.input_fn, max_steps=next_checkpoint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2452, in t
rain
    saving_listeners=saving_listeners)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 358, in tr
ain
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 1124, in _
train_model
    return self._train_model_default(input_fn, hooks, saving_listeners)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 1154, in _
train_model_default
    features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2251, in _
call_model_fn
    config)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 1112, in _
call_model_fn
    model_fn_results = self._model_fn(features=features, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2547, in _
model_fn
    input_holders.generate_infeed_enqueue_ops_and_dequeue_fn())
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1167, in g
enerate_infeed_enqueue_ops_and_dequeue_fn
    self._invoke_input_fn_and_record_structure())
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1243, in _
invoke_input_fn_and_record_structure
    self._inputs_structure_recorder, host_device, host_id))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 830, in ge
nerate_per_host_v2_enqueue_ops_fn_for_host
    inputs = _Inputs.from_input_fn(input_fn(user_context))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2423, in _
input_fn
    return input_fn(**kwargs)
  File "/home/rubensvectomobile/tpu/models/official/efficientnet/imagenet_input.py", line 161, in input_fn
    dataset = self.make_source_dataset(current_host, num_hosts)
  File "/home/rubensvectomobile/tpu/models/official/efficientnet/imagenet_input.py", line 275, in make_source_datas
et
    dataset = tf.data.Dataset.list_files(file_pattern, shuffle=False)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 1484, in list_files
    return DatasetV1Adapter(DatasetV2.list_files(file_pattern, shuffle, seed))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 697, in list_files
    condition, [message], summarize=1, name="assert_not_empty")
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/tf_should_use.py", line 193, in wrapped
    return _add_should_use_warning(fn(*args, **kwargs))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 160, in Assert
    return gen_logging_ops._assert(condition, data, summarize, name="Assert")
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_logging_ops.py", line 72, in _assert
    name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_
op_helper
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
    self._traceback = tf_stack.extract_stack()
InvalidArgumentError (see above for traceback): assertion failed: [No files matched pattern: gs://cls-loc/train-*]
         [[node input_pipeline_task0/list_files/assert_not_empty/Assert (defined at /home/rubensvectomobile/tpu/mod
els/official/efficientnet/imagenet_input.py:275) ]]

What is the directory structure to feed EfficientNet ? Is it somewhat related to this, containing JPEGs ?

# folders
train  val

$ ls train
n01440764  n01943899  n02105056  n02361337  n02843684  n03379051

If so, where are the dependent variables (target) placed ? Are they the synset_mapping.txt ?

Thanks in advance.

( for more info about my efforts, access: EfficientNet )

saberkun commented 5 years ago

Hi, all training data are tfrecords. You need to get a directory from https://github.com/tensorflow/tpu/blob/master/tools/datasets/imagenet_to_gcs.py or make your training data like that. E.g. gs://cloud-tpu-test-datasets/fake_imagenet