tfds errors with tfds.testing.mock_data

[mattiasmar]

What I need help with / What I was wondering How to use tfds.testing.mock_data correctly

What I've tried so far This code on its own works fine:

import tensorflow_datasets as tfds
ds_builder = tfds.builder('mnist')
ds_builder.download_and_prepare()
train_ds = tfds.as_numpy(ds_builder.as_dataset(split='train', batch_size=-1))

When decorating that snippet with tfds.testing.mock_data I get errors:

with tfds.testing.mock_data(num_examples=8, data_dir='/workdisk/.tfds2/metadata'):
  import tensorflow_datasets as tfds
  ds_builder = tfds.builder('mnist')
  ds_builder.download_and_prepare()
  train_ds = tfds.as_numpy(ds_builder.as_dataset(split='train', batch_size=-1))

Error trace:

I1220 13:56:57.120099 140470378489664 mocking.py:162] Metadata NOT found for mnist at /workdisk/jax/FlaxExamples/flax/.tfds/metadata/mnist/3.0.1. Will use `MockPolicy.USE_CODE.`
I1220 13:56:57.121221 140470378489664 logging_logger.py:35] Constructing tf.data.Dataset mnist for split train, from /workdisk/jax/FlaxExamples/flax/.tfds/metadata/mnist/3.0.1
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/tensorflow_datasets/core/dataset_builder.py:622: get_single_element (from tensorflow.python.data.experimental.ops.get_single_element) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.get_single_element()`.
W1220 13:56:57.204224 140470378489664 deprecation.py:341] From /usr/local/lib/python3.8/dist-packages/tensorflow_datasets/core/dataset_builder.py:622: get_single_element (from tensorflow.python.data.experimental.ops.get_single_element) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.get_single_element()`.
terminate called after throwing an instance of 'std::length_error'
  what():  vector::reserve
Fatal Python error: Aborted

Environment information Latest Nvidia dev docker image: nvidia/cuda:11.4.2-cudnn8-devel-ubuntu20.04

• Operating System: ubuntu20.04
• Python version: 3.8
• tensorflow 2.7.0
• tensorflow-datasets 4.4.0
• tensorflow-estimator 2.7.0
• tensorflow-io-gcs-filesystem 0.23.1
• tensorflow-metadata 1.5.0

[tomvdw]

Hi Mattias,

Thanks for reporting this! This is already fixed in tf-nightly. We'll soon release a new version of TFDS so that this warning should disappear automatically.

Kind regards, Tom

[mattiasmar]

Hi @tomvdw

I tested now with the tensorflow nightly docker image tensorflow/tensorflow:nightly-gpu-jupyter and I get the same error.

________                               _______________
___  __/__________________________________  ____/__  /________      __
__  /  _  _ \_  __ \_  ___/  __ \_  ___/_  /_   __  /_  __ \_ | /| / /
_  /   /  __/  / / /(__  )/ /_/ /  /   _  __/   _  / / /_/ /_ |/ |/ /
/_/    \___//_/ /_//____/ \____//_/    /_/      /_/  \____/____/|__/

You are running this container as user with ID 1000 and group 1001,
which should map to the ID and group for your user on the Docker host. Great!

tf-docker /workdisk > ipython
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.30.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import tensorflow_datasets as tfds
   ...: with tfds.testing.mock_data(num_examples=8, data_dir='/workdisk/.tfds2/metadata'):
   ...:     import tensorflow_datasets as tfds
   ...:     ds_builder = tfds.builder('mnist')
   ...:     ds_builder.download_and_prepare()
   ...:     train_ds = tfds.as_numpy(ds_builder.as_dataset(split='train', batch_size=-1))
   ...:
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/tensorflow_datasets/core/dataset_builder.py:622: get_single_element (from tensorflow.python.data.experimental.ops.get_single_element) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.get_single_element()`.
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/tensorflow_datasets/core/dataset_builder.py:622: get_single_element (from tensorflow.python.data.experimental.ops.get_single_element) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.get_single_element()`.
terminate called after throwing an instance of 'std::length_error'
  what():  vector::reserve
Aborted (core dumped)

[mattiasmar]

This is the Dockerfile that I'm using: Dockerfile_TF_nightly_with_tfds.txt

[mattiasmar]

If your intention with "nightly" was to install tfds with pip install tfds-nightly then the result was that I still get an error, but not the warnings:

terminate called after throwing an instance of 'std::length_error'
  what():  vector::reserve
Aborted (core dumped)

[mattiasmar]

In contrast this works without problem:

 import tensorflow_datasets as tfds
 with tfds.testing.mock_data(num_examples=5):
    ds = tfds.load('imagenette', split='train')  #'mnist' works too
    for ex in ds:  # ds will yield randomly generated examples.
    print(ex['image'].shape)