OSError: train.idx not found

Hey @wwMark,

I'm running a vanilla python run_model.py and the code cannot find train.idx which makes sense as I don't have it. I got the data using your script ie bash download_dataset.sh flag_simple data and my data/ folder contains

$ ls data/flag_simple                                                                                                                          
meta.json  test.tfrecord  train.tfrecord  valid.tfrecord

Any idea what I might have done wrong?

Full Tracebak

``` python run_model.py I0927 11:29:48.498236 140059436225088 run_model.py:270] Program started at time Mon Sep 27 11:29:38 2021 2021-09-27 11:29:48,498 - Program started at time Mon Sep 27 11:29:38 2021 I0927 11:29:48.498435 140059436225088 run_model.py:274] Start training...... 2021-09-27 11:29:48,498 - Start training...... /home/mila/s/schmidtv/.conda/envs/fluids2/lib/python3.8/site-packages/torch/nn/modules/lazy.py:175: UserWarning: Lazy modules are a new feature under heavy development so changes to the API or functionality can happen at any moment. warnings.warn('Lazy modules are a new feature under heavy development ' Traceback (most recent call last): File "run_model.py", line 321, in app.run(main) File "/home/mila/s/schmidtv/.conda/envs/fluids2/lib/python3.8/site-packages/absl/app.py", line 312, in run _run_main(main, args) File "/home/mila/s/schmidtv/.conda/envs/fluids2/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main sys.exit(main(argv)) File "run_model.py", line 281, in main train_loss_record = learner(params, model) File "run_model.py", line 121, in learner ds_loader = dataset.load_dataset(FLAGS.dataset_dir, 'train', batch_size=batch_size, prefetch_factor=prefetch_factor, File "/home/mila/s/schmidtv/fluids/meshgraphnets-vict0rsch/dataset.py", line 114, in load_dataset return DataLoader(FlagSimpleDataset(path=path, split=split, add_targets=add_targets, split_and_preprocess=split_and_preprocess), batch_size=batch_size, prefetch_factor=prefetch_factor, shuffle=True, num_workers=0)# , collate_fn=collate_fn) File "/home/mila/s/schmidtv/fluids/meshgraphnets-vict0rsch/migration_utilities/flag_simple_torch_dataset.py", line 46, in __init__ self.dataset = list(iter(loader)) File "/home/mila/s/schmidtv/.conda/envs/fluids2/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in __next__ data = self._next_data() File "/home/mila/s/schmidtv/.conda/envs/fluids2/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 557, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/home/mila/s/schmidtv/.conda/envs/fluids2/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 28, in fetch data.append(next(self.dataset_iter)) File "/home/mila/s/schmidtv/.conda/envs/fluids2/lib/python3.8/site-packages/tfrecord/reader.py", line 219, in example_loader for record in record_iterator: File "/home/mila/s/schmidtv/.conda/envs/fluids2/lib/python3.8/site-packages/tfrecord/reader.py", line 82, in tfrecord_iterator index = np.loadtxt(index_path, dtype=np.int64)[:, 0] File "/home/mila/s/schmidtv/.conda/envs/fluids2/lib/python3.8/site-packages/numpy/lib/npyio.py", line 1065, in loadtxt fh = np.lib._datasource.open(fname, 'rt', encoding=encoding) File "/home/mila/s/schmidtv/.conda/envs/fluids2/lib/python3.8/site-packages/numpy/lib/_datasource.py", line 194, in open return ds.open(path, mode, encoding=encoding, newline=newline) File "/home/mila/s/schmidtv/.conda/envs/fluids2/lib/python3.8/site-packages/numpy/lib/_datasource.py", line 531, in open raise IOError("%s not found." % path) OSError: /home/mila/s/schmidtv/fluids/meshgraphnets-vict0rsch/data/flag_simple/train.idx not found. ```

wwMark / meshgraphnets

OSError: train.idx not found #2