Closed yCobanoglu closed 2 weeks ago
I also have a problem with the following code:
import tensorflow_datasets as tfds # TFDS for MNIST
import tensorflow as tf # TensorFlow operations
def process_sample(sample):
image, label = sample['image'], sample['label']
image = tf.cast(image, tf.float32) / 255.
image = image.numpy()
return {'image': image, 'label': label}
def get_mnist(num_epochs, batch_size):
"""Load MNIST train and test datasets into memory."""
train_ds = tfds.load('mnist', split='train')
test_ds = tfds.load('mnist', split='test')
train_ds = train_ds.map(process_sample)
test_ds = test_ds.map(process_sample)
# create shuffled dataset by allocating a buffer size of 1024 to randomly draw elements from
train_ds = train_ds.repeat(num_epochs).shuffle(1024)
# group into batches of batch_size and skip incomplete batch, prefetch the next sample to improve latency
train_ds = train_ds.batch(batch_size, drop_remainder=True).prefetch(1)
# create shuffled dataset by allocating a buffer size of 1024 to randomly draw elements from
test_ds = test_ds.shuffle(1024)
# group into batches of batch_size and skip incomplete batch, prefetch the next sample to improve latency
test_ds = test_ds.batch(batch_size, drop_remainder=True).prefetch(1)
return train_ds, test_ds
throws:
File "/home/yunus/PycharmProjects/DeepLearning1/src/datasets.py", line 7, in process_sample *
image = image.numpy()
AttributeError: 'SymbolicTensor' object has no attribute 'numpy'
Hi @yCobanoglu , thanks for contacting us and sorry for the late reply.
tfds.as_numpy
can't be called on a SymbolicTensor directly...
Have you tried following the function's docstring with something like:
train_ds = tfds.load('mnist', split='train')
train_ds = tfds.as_numpy(train_ds)
Hi @yCobanoglu , I assume the suggestion above worked? I'm closing the issue for now, feel free to reopen it if you encounter any further problem!
Short description
tfds.as_numpy
not working correctly.Additional context After following the flax tutorial in step 2 the mnist data is loaded. It requires you to install tensorflow just to use a single function, namely
tf.cast
. I tried to get rid of the tensorflow dependency and stumbled upon this bug.Environment information
Operating System: Linux
Python version: 3.12.4
tensorflow-datasets
/tfds-nightly
version: 4.9.6tensorflow
/tf-nightly
version: 2.17.0Problem exists with tfds-nightly version.
Reproduction instructions
Error