meshgraphcnn: issue with some of the datasets (e.g. sphere_dynamic, flag_dynamic)

[oshrihalimi1]

This line: https://github.com/deepmind/deepmind-research/blob/6fcb84268e74af981ae1496bfc2cb9ba9d701ef2/meshgraphnets/dataset.py#L52 returns an empty dataset when providing the path to the .tfrecord files.

The data files were downloaded using the following command: bash meshgraphnets/download_dataset.sh sphere_dynamic $DATA_FOLDER

For flag_simple this line works, so there might be an issue with the files (?)

[alvarosg]

@tobiaspfaff

[tobiaspfaff]

The raw dataframe might appear to be empty due to the variable-length encoding. However, using the dataset loader you should be able to load this dataset. I just tried, and got this output:

ds = meshgraphnets.dataset.load_dataset('meshgraphnets/data/sphere_dynamic', 'train') ds <DatasetV1Adapter shapes: {node_type: (?, ?, 1), world_pos: (?, ?, 3), cells: (?, ?, 3), mesh_pos: (?, ?, 2)}, types: {node_type: tf.int32, world_pos: tf.float32, cells: tf.int32, mesh_pos: tf.float32}>

Is this different from what you get?

[oshrihalimi1]

I get this: ds <TFRecordDatasetV1 shapes: (), types: tf.string>

Then, the training code fails with the following error:

Traceback (most recent call last):
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\absl\app.py", line 308, in run
    _run_main(main, args)
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\absl\app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "C:/Users/oshrihalimi/Projects/deepmind-research/meshgraphnets/run_model.py", line 125, in main
    learner(model, params)
  File "C:/Users/oshrihalimi/Projects/deepmind-research/meshgraphnets/run_model.py", line 60, in learner
    noise_gamma=params['gamma'])
  File "C:\Users\oshrihalimi\Projects\deepmind-research\meshgraphnets\dataset.py", line 85, in split_and_preprocess
    ds = ds.map(add_noise, num_parallel_calls=8)
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 1913, in map
    self, map_func, num_parallel_calls, preserve_cardinality=False))
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 3472, in __init__
    use_legacy_function=use_legacy_function)
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 2713, in __init__
    self._function = wrapper_fn._get_concrete_function_internal()
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\eager\function.py", line 1853, in _get_concrete_function_internal
    *args, **kwargs)
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\eager\function.py", line 1847, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\eager\function.py", line 2147, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\eager\function.py", line 2038, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\framework\func_graph.py", line 915, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 2707, in wrapper_fn
    ret = _wrapper_helper(*args)
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 2648, in _wrapper_helper
    self._input_structure, args)
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\data\util\structure.py", line 226, in from_compatible_tensor_list
    element_spec, tensor_list)
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\data\util\structure.py", line 201, in _from_tensor_list_helper
    flat_ret.append(decode_fn(component_spec, value))
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\data\util\structure.py", line 225, in <lambda>
    lambda spec, value: spec._from_compatible_tensor_list(value),
  File "C:\Users\oshrihalimi\Anaconda3\envs\openai\lib\site-packages\tensorflow_core\python\ops\ragged\ragged_tensor.py", line 2027, in _from_compatible_tensor_list
    "ragged_rank must be non-negative; got %s." % self._ragged_rank)
ValueError: ragged_rank must be non-negative; got 0.
python-BaseException

[CatMoonLiao]

hello, i tried to train MGN on sphere_dynamic dataset. I can successfully load the tfrecord file, but still stuck in the same error. After doing ds = ds.flat_map(tf.data.Dataset.from_tensor_slices) in line85 of dataset.py, the ragged_rank reduce to 0. Maybe is because flatten RaggedTensorSpec may get wrong result? Hope someone can help! Thanks!