Converting Mine DROID Dataset to RLDS fail

Jinming-Li commented 8 months ago

The sov data I collected in my own environment cannot be processed directly after being converted into MP4 format. I can use the code you provided to process it. @https://github.com/kpertsch/droid_dataset_builder
I would like to ask what kind of processing you have done on the generated MP4 format files. I found that 27TB The mp4 format of the data is H.264, and mine is mpeg4.
At the same time, my own duration will be longer than the original data you provided. What kind of processing did you perform?

kpertsch commented 8 months ago

Please consider keeping the titles of your issues short & descriptive, so they are easier for others to parse that may have the same issue.

Regarding your issue, could you please provide more information about how your conversion script fails? Is there any error message you can send?

Jinming-Li commented 8 months ago

I use the code https://github.com/AlexanderKhazatsky/DROID/blob/main/scripts/convert/svo_to_mp4.py to deal svo data . when run tfds build --overwrite to deal MP4 data. I get

INFO[build.py]: Loading dataset  from path: /home/eai/Dev-Code/droid_dataset_builder/droid/droid.py
2024-04-03 17:59:53.706128: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-04-03 17:59:53.755267: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-04-03 17:59:54.427595: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-04-03 17:59:54.731664: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1960] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
INFO[dataset_info.py]: Load dataset info from /home/eai/tensorflow_datasets/droid/1.0.0
2024-04-03 18:00:09.477083: W tensorflow/tsl/platform/cloud/google_auth_provider.cc:184] All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "NOT_FOUND: Could not locate the credentials file.". Retrieving token from GCE failed with "ABORTED: All 10 retry attempts failed. The last failure: Error executing an HTTP request: HTTP response code 503".
INFO[build.py]: download_and_prepare for dataset droid/1.0.0...
INFO[native_type_compatibility.py]: Using Any for unsupported type: typing.Sequence[~T]
INFO[bigquery.py]: No module named google.cloud.bigquery_storage_v1. As a result, the ReadFromBigQuery transform *CANNOT* be used with `method=DIRECT_READ`.
INFO[dataset_builder.py]: Generating dataset droid (/home/eai/tensorflow_datasets/droid/1.0.0)
Downloading and preparing dataset Unknown size (download: Unknown size, generated: Unknown size, total: Unknown size) to /home/eai/tensorflow_datasets/droid/1.0.0...
Crawling all episode paths...
dirname!! /home/eai/Dev-Code/droid_dataset_builder/2024-04-03
dirname!! /home/eai/Dev-Code/droid_dataset_builder/2024-04-03/Wed_Apr__3_17-02-51_2024
dirname!! /home/eai/Dev-Code/droid_dataset_builder/2024-04-03/Wed_Apr__3_17-04-07_2024
dirname!! /home/eai/Dev-Code/droid_dataset_builder/2024-04-03/Wed_Apr__3_17-00-11_2024
Found 3 episodes!
Crawling all episode paths...
dirname!! /home/eai/Dev-Code/droid_dataset_builder/2024-04-03
dirname!! /home/eai/Dev-Code/droid_dataset_builder/2024-04-03/Wed_Apr__3_17-02-51_2024
dirname!! /home/eai/Dev-Code/droid_dataset_builder/2024-04-03/Wed_Apr__3_17-04-07_2024
dirname!! /home/eai/Dev-Code/droid_dataset_builder/2024-04-03/Wed_Apr__3_17-00-11_2024
Found 3 episodes!
Generating splits...:   0%|                                                                                                                                       | 0/1 [00:00<?, ? splits/s]Generating with 10 workers!
Processing chunk 1 of 1.
                                                              Skipping trajectory because there was an error in data processing for /home/eai/Dev-Code/droid_dataset_builder/2024-04-03/Wed_Apr__3_17-04-07_2024.                                          
                                                                  Skipping trajectory because there was an error in data processing for /home/eai/Dev-Code/droid_dataset_builder/2024-04-03/Wed_Apr__3_17-02-51_2024.                                          
                                                                  Skipping trajectory because there was an error in data processing for /home/eai/Dev-Code/droid_dataset_builder/2024-04-03/Wed_Apr__3_17-00-11_2024.                                          
                                                                  Writing conversion results...
Finishing split conversion...                                     
Traceback (most recent call last):                                                                                                                                                           
  File "/home/eai/miniconda3/envs/rlds_env/bin/tfds", line 8, in <module>
    sys.exit(launch_cli())
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/main.py", line 109, in launch_cli
    app.run(main, flags_parser=_parse_flags)
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/main.py", line 104, in main
    args.subparser_fn(args)
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/build.py", line 312, in _build_datasets
    process_builder_fn(builder)
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/build.py", line 551, in _download_and_prepare
    builder.download_and_prepare(
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/tensorflow_datasets/core/logging/__init__.py", line 166, in __call__
    return function(*args, **kwargs)
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/tensorflow_datasets/core/dataset_builder.py", line 691, in download_and_prepare
    self._download_and_prepare(
  File "/home/eai/Dev-Code/droid_dataset_builder/droid/tfds_utils.py", line 88, in _download_and_prepare
    future = split_builder.submit_split_generation(
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/tensorflow_datasets/core/split_builder.py", line 341, in submit_split_generation
    return self._build_from_generator(**build_kwargs)
  File "/home/eai/Dev-Code/droid_dataset_builder/droid/tfds_utils.py", line 199, in _build_from_generator
    shard_lengths, total_size = writer.finalize()
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/tensorflow_datasets/core/writer.py", line 245, in finalize
    shard_specs = _get_shard_specs(
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/tensorflow_datasets/core/writer.py", line 113, in _get_shard_specs
    shard_boundaries = _get_shard_boundaries(num_examples, num_shards)
  File "/home/eai/miniconda3/envs/rlds_env/lib/python3.9/site-packages/tensorflow_datasets/core/writer.py", line 144, in _get_shard_boundaries
    raise AssertionError("No examples were yielded.")
AssertionError: No examples were yielded.

kpertsch commented 8 months ago

You may want to comment out these try...catch statements here to more clearly show what the underlying error is: https://github.com/kpertsch/droid_dataset_builder/blob/8c3f83f3dfa490e670002886907f96914c5961a2/droid/droid.py#L42

Jinming-Li commented 8 months ago

Thanks.

droid-dataset / droid_policy_learning

Converting Mine DROID Dataset to RLDS fail #3