search

torchmd / torchmd-net

Training neural network potentials
MIT License
335 stars 75 forks source link

Refactoring memmapped dataset classes #256

Closed stefdoerr closed 10 months ago

stefdoerr commented 10 months ago

I refactored all memmaped datasets into a separate class to reduce code duplication

RaulPPelaez commented 10 months ago

That is really cool! Thanks!

RaulPPelaez commented 10 months ago

Lets see if tests pass and then merge

RaulPPelaez commented 10 months ago 
 tests/test_dataset_comp6.py::test_dataset_s66x8 FAILED
 ______________________________ test_dataset_s66x8 ______________________________

    def test_dataset_s66x8():

        with TemporaryDirectory() as root:
>           data_set = S66X8(root)

tests/test_dataset_comp6.py:15: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/share/miniconda3/envs/torchmd-net/lib/python3.11/site-packages/torchmdnet/datasets/comp6.py:41: in __init__
    super().__init__(
/usr/share/miniconda3/envs/torchmd-net/lib/python3.11/site-packages/torchmdnet/datasets/memdataset.py:47: in __init__
    super().__init__(root, transform, pre_transform, pre_filter)
/usr/share/miniconda3/envs/torchmd-net/lib/python3.11/site-packages/torch_geometric/data/dataset.py:102: in __init__
    self._process()
/usr/share/miniconda3/envs/torchmd-net/lib/python3.11/site-packages/torch_geometric/data/dataset.py:228: in _process
    if files_exist(self.processed_paths):  # pragma: no cover
/usr/share/miniconda3/envs/torchmd-net/lib/python3.11/site-packages/torch_geometric/data/dataset.py:187: in processed_paths
    files = self.processed_file_names
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'S66X8' object has no attribute 'idx_mm'") raised in repr()] S66X8 object at 0x7f16715fe310>

    @property
    def processed_file_names(self) -> Union[str, List[str], Tuple]:
        r"""The name of the files in the :obj:`self.processed_dir` folder that
        must be present in order to skip processing."""
>       raise NotImplementedError
E       NotImplementedError

/usr/share/miniconda3/envs/torchmd-net/lib/python3.11/site-packages/torch_geometric/data/dataset.py:58: NotImplementedError
guillemsimeon commented 10 months ago

Cool!

stefdoerr commented 10 months ago

I see, there is a bit of a chicken-egg issue with the processed_file_names. I'll fix it

RaulPPelaez commented 10 months ago

You got it, shall we merge then?