datasets

[casperdcl]

something like this[^keras] (for use in notebook demos such as https://github.com/TomographicImaging/CIL-Demos/pull/144):

data = cil.datasets.walnut.load()

btw internal logic could be roughly:

class Dataset:
    cache = os.getenv("CIL_DATASETS", "~/.cache/cil")
    @classmethod
    def load(cls):
        filename = brainweb.get_file(
            cls.source.rsplit("/", 1)[-1],
            cls.source,
            cache_dir=cls.cache)
        if filename.endswith(".zip"):
            return cls.load_from_zip(filename)
        raise ValueError(f"{filename}: unknown extension")

class walnut(Dataset):
    source = "https://zenodo.org/record/4822516/files/walnut.zip"

[^keras]: loosely inspired by https://keras.io/api/datasets approach

[paskino]

We have a structure for data which could be used.

https://github.com/TomographicImaging/CIL/blob/db5a2a6cd3bddfbbf53e65f0549ac206096e5b44/Wrappers/Python/cil/utilities/dataexample.py#L39-L47