tensorflow / io

Dataset, streaming, and file system extensions maintained by TensorFlow SIG-IO
Apache License 2.0
697 stars 282 forks source link

TF2.16.2 with tensorflow-io 0.37.1 shows pure virtual method called, terminate called without an active exception causing exit code 134 when working with s3 filesystem #2039

Open shantanutrip opened 1 month ago

shantanutrip commented 1 month ago

Hi,

When trying to use s3 filesystem with tensorflow-io==0.37.1 and tensorflow==2.16.2, I get the following error:

pure virtual method called
terminate called without an active exception
Aborted (core dumped)

This exits with error code 134.

This was discussed in another issue: https://github.com/tensorflow/io/issues/1912 for TF2.14 which was closed earlier. However, since the issue still persists, I am reopening it for resolution.

Steps to Reproduce:

  1. docker pull tensorflow/tensorflow:2.16.2-gpu (Image details from tf dockerhub)
  2. Exec into the container and pip install tensorflow-io==0.37.1
  3. Create a dummy record within the container using the following script:
    
    import tensorflow as tf
    import numpy as np

example_path = "example.tfrecords" np.random.seed(0)

with tf.io.TFRecordWriter(example_path) as filewriter: for in range(4): x, y = np.random.random(), np.random.random()

record_bytes = tf.train.Example(features=tf.train.Features(feature={
    "x": tf.train.Feature(float_list=tf.train.FloatList(value=[x])),
    "y": tf.train.Feature(float_list=tf.train.FloatList(value=[y])),
})).SerializeToString()
file_writer.write(record_bytes)
Source: https://www.tensorflow.org/api_docs/python/tf/io/TFRecordWriter
4. Upload the example.tfrecords file to s3 bucket
5. Use the following script with the s3 location of the uploaded record file:

import tensorflow as tf import tensorflow_io as tfio s3_filename="" #Fill this with the s3 uri of the uploaded file. Should look like s3://object-name/example.tfrecords ds = tf.data.TFRecordDataset(s3_filename) for element in ds: print(f"{element.shape}")

Script motivated from previous issue: https://github.com/tensorflow/io/issues/1912

While trying to install any other versions of `tensorflow-io` like : `0.32.*`, `0.33.*`,`0.34.*`,`0.35.*` or `0.36.*`, I get the following error:

File system scheme 's3' not implemented



Fix needed for `Aborted (core dumped)` issue. Thanks!
rivershah commented 1 month ago

This is a duplicate. Please fix https://github.com/tensorflow/io/issues/1912 The issue is closed but the problem is not resolved

shantanutrip commented 1 month ago

Hi team, any updates on this issue?

Please provide resolution for the common problem underlying both the issues - this and/or https://github.com/tensorflow/io/issues/1912.

Thanks!