jsawruk commented 7 years ago

System information

What is the top-level directory of the model you are using: tensorflow/models/object_detection
Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Mac OS 10.12.5
TensorFlow installed from (source or binary): Binary (pip install within conda)
TensorFlow version (use command below): 1.2.0
Bazel version (if compiling from source): 0.4.5
CUDA/cuDNN version: N/A
GPU model and memory: N/A

Exact command to reproduce:

python object_detection/create_pet_tf_record.py     \
--label_map_path=object_detection/data/pet_label_map.pbtxt     \
--data_dir=`pwd`     \
--output_dir=`pwd`

Describe the problem

When following the tutorial instructions Quick Start: Distributed Training on the Oxford-IIIT Pets Dataset on Google Cloud (https://github.com/tensorflow/models/blob/master/object_detection/g3doc/running_pets.md), the above code may not work depending on where the annotations.tar.gz and images.tar.gz files are located.

Under the heading Getting the Oxford-IIIT Pets Dataset and Uploading it to Google Cloud Storage, there are two code blocks. The first lists a directory structure, while the second lists commands to execute from the tensorflow/models directory.

The first code block (directory structure) looks like this:

+ object_detection/
  + data/
  - images.tar.gz
  - annotations.tar.gz
  - create_pet_tf_record.py
  ... other files and directories

This shows that images.tar.gz and annotations.tar.gz are located within the the directory tensorflow/models/object_detection. However, when running the above Python command (located in the next block), the following error occurs:

tensorflow.python.framework.errors_impl.NotFoundError: /tensorflow/models/annotations/trainval.txt

This occurs because the script is looking for the files in tensorflow/models, not tensorflow/models/object_detection.

If the directory structure is ignored, and the commands are run, then no error occurs because the files are in the correct place. Similarly, directions above the directory structure say:

You will need to download both the image dataset images.tar.gz and the groundtruth data annotations.tar.gz to the tensorflow/models directory.

This is a documentation error. The directory structure is incorrect and/or confusing. I initially downloaded the files into tensorflow/models as the above text directed, but then moved them into tensorflow/models/object_detection because the directions indicated that's what my directory structure should look like.

I propose changing the directory structure to this:

+ models/
  - annotations.tar.gz
  - images.tar.gz
  + object_detection/
    + data/
    - create_pet_tf_record.py
    ... other files and directories

I find this directory structure to be less confusing, as it shows the annotations.tar.gz and images.tar.gz to be contained within the tensorflow/models directory, rather than tensorflow/models/object_detection.

Source code / logs

Traceback (most recent call last):
  File "object_detection/create_pet_tf_record.py", line 213, in <module>
    tf.app.run()
  File "/anaconda/envs/tf/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "object_detection/create_pet_tf_record.py", line 192, in main
    examples_list = dataset_util.read_examples_list(examples_path)
  File "/tensorflow/models/object_detection/utils/dataset_util.py", line 59, in read_examples_list
    lines = fid.readlines()
  File "/anaconda/envs/tf/lib/python3.5/site-packages/tensorflow/python/lib/io/file_io.py", line 181, in readlines
    self._preread_check()
  File "/anaconda/envs/tf/lib/python3.5/site-packages/tensorflow/python/lib/io/file_io.py", line 78, in _preread_check
    compat.as_bytes(self.__name), 1024 * 512, status)
  File "/anaconda/envs/tf/lib/python3.5/contextlib.py", line 66, in __exit__
    next(self.gen)
  File "/anaconda/env/tf/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.NotFoundError: /tensorflow/models/annotations/trainval.txt