Error evaluating the sun397 dataset

[mseitzer]

Hi,

I am running into the following error when evaluating the sun397 dataset: RuntimeError: Could not infer dtype of NoneType. I already re-downloaded the dataset to make sure there are not any data issues. I am able to successfully evaluate other datasets.

Any idea what might go wrong?

The full stacktrace:

Traceback (most recent call last):
  File "<path_to_env>/bin/unibench", line 8, in <module>
    sys.exit(run())
    │        └ <function run at 0x7fc90fb5ed40>
    └ <module 'sys' (built-in)>
  File "<path_to_code>/unibench/unibench/evaluator.py", line 363, in run
    fire.Fire(Evaluator)
    │         └ <class 'unibench.evaluator.Evaluator'>
    └ <module 'fire' from '<path_to_env>/lib/python3.11/site-packag...
  File "<path_to_env>/lib/python3.11/site-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      │     │          │     │                 │        └ 'unibench'
                      │     │          │     │                 └ {}
                      │     │          │     └ Namespace(verbose=False, interactive=False, separator='-', completion=None, help=False, trace=False)
                      │     │          └ ['--benchmarks_dir=/fsx-dino/data/seitzer/data/unibench', '--output-dir=/fsx-dino/data/seitzer/results/unibench', 'evaluate', '-...
                      │     └ <class 'unibench.evaluator.Evaluator'>
                      └ <function _Fire at 0x7fc90fbe6fc0>
  File "<path_to_env>/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "<path_to_env>/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
    │           │   │          └ {}
    │           │   └ [1000, False, 'cuda', 32]                                                                                                                                                       [0/1852]
    │           └ <bound method Evaluator.evaluate of <unibench.evaluator.Evaluator object at 0x7fc900056c90>>
    └ <bound method Evaluator.evaluate of <unibench.evaluator.Evaluator object at 0x7fc900056c90>>
  File "<path_to_code>/unibench/unibench/evaluator.py", line 322, in evaluate
    for idx, batch in enumerate(dl):
  File "<path_to_env>/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 701, in __next__
    data = self._next_data()
           └ <torch.utils.data.dataloader._MultiProcessingDataLoaderIter object at 0x7fc67f0ff310>
  File "<path_to_env>/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 1445, in _next_data
    return self._process_data(data)
           │                  └ <torch._utils.ExceptionWrapper object at 0x7fc67e0e4590>
           └ <torch.utils.data.dataloader._MultiProcessingDataLoaderIter object at 0x7fc67f0ff310>
  File "<path_to_env>/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 1491, in _process_data
    data.reraise()
    └ <torch._utils.ExceptionWrapper object at 0x7fc67e0e4590>
  File "<path_to_env>/lib/python3.11/site-packages/torch/_utils.py", line 733, in reraise
    raise exception
          └ RuntimeError('Caught RuntimeError in DataLoader worker process 40.\nOriginal Traceback (most recent call last):\n  File "/opt/hp...
RuntimeError: Caught RuntimeError in DataLoader worker process 40.
Original Traceback (most recent call last):
  File "<path_to_env>/lib/python3.11/site-packages/torch/utils/data/_utils/worker.py", line 351, in _worker_loop
    data = fetcher.fetch(index)  # type: ignore[possibly-undefined]
           ^^^^^^^^^^^^^^^^^^^^
  File "<path_to_env>/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 55, in fetch
    return self.collate_fn(data)
           ^^^^^^^^^^^^^^^^^^^^^
  File "<path_to_env>/lib/python3.11/site-packages/torch/utils/data/_utils/collate.py", line 398, in default_collate
    return collate(batch, collate_fn_map=default_collate_fn_map)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path_to_env>/lib/python3.11/site-packages/torch/utils/data/_utils/collate.py", line 211, in collate
    return [
           ^
  File "<path_to_env>/lib/python3.11/site-packages/torch/utils/data/_utils/collate.py", line 212, in <listcomp>
    collate(samples, collate_fn_map=collate_fn_map)
  File "<path_to_env>/lib/python3.11/site-packages/torch/utils/data/_utils/collate.py", line 155, in collate
    return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path_to_env>/lib/python3.11/site-packages/torch/utils/data/_utils/collate.py", line 309, in collate_int_fn
    return torch.tensor(batch)
           ^^^^^^^^^^^^^^^^^^^
RuntimeError: Could not infer dtype of NoneType

[haideraltahan]

Hi @mseitzer,

I'll look into it and get back to you. Thank you for reporting this.