search

waymo-research / waymo-open-dataset

Waymo Open Dataset
https://www.waymo.com/open
Other
2.69k stars 611 forks source link

Number of scenarios within one tf-example file #714

Open ArielleZhang opened 1 year ago

ArielleZhang commented 1 year ago

Hi,

I just want to clarify that is there only one scenario within one tf-example file for waymo_open_dataset_motion_v_1_2_0 if the type of the file is tf-example?

Appreciate your help!

scott-ettinger commented 1 year ago

Hi, Each file contains many scenarios in tfrecord format stored as serialized tensorflow example protocol buffer objects.

Please let me know if you have further questions.

ArielleZhang commented 1 year ago

Hi,

Thanks for the help, I recommend to change the motion tutorial since it loads only one single scenario from the parsed tf.record file, which is a little bit confusing.

Thanks!

scott-ettinger commented 1 year ago

Thank you for your feedback!

miladsasha commented 6 months ago

I am unable to run any .tf record using the following part of the code-in-motion tutorial can you help please:

part of code: Create Dataset.

dataset = tf.data.TFRecordDataset(FILENAME, compression_type='') data = next(dataset.as_numpy_iterator()) parsed = tf.io.parse_single_example(data, features_description)

error: InvalidArgumentError Traceback (most recent call last)

in () 42 43 # Print the available features in the first example ---> 44 for example in parsed_dataset.take(1): # Taking just one example for demonstration 45 available_features = [key for key, value in example.items() if value.numpy() is not None] 46 print("Available features:") 3 frames /usr/local/lib/python3.10/dist-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name) 7213 def raise_from_not_ok_status(e, name): 7214 e.message += (" name: " + name if name is not None else "") -> 7215 raise core._status_to_exception(e) from None # pylint: disable=protected-access 7216 7217 InvalidArgumentError: {{function_node __wrapped__IteratorGetNext_output_types_20_device_/job:localhost/replica:0/task:0/device:CPU:0}} Feature: roadgraph_samples/dir (data type: string) is required but could not be found. [[{{node ParseSingleExample/ParseExample/ParseExampleV2}}]] [Op:IteratorGetNext] the roadgraph_samples/dir was available among the features, I tested it for all versions of the .tf motion dataset. Also when I eliminate that feature, another feature turns into an error