data loader structure

[davidsebfischer]

Components of a data loader are most likely:

• a yaml file describing meta data. in single-cell resolved data, you can distinguish cell wise and data set wise meta data, we directly write sample wise meta data into the yaml and add the key of the obs column describing cell wise meta data in a corresponding entry in the yaml file. it may be that you can drop cell wise data here for some modalities, otherwise i would recommond adopting almost the exact same yaml, that would also help interoperability with sfaira and cellxgene data
• a function loading a defined object, for us adata, here you have to decide what that is
• a class that collects all written loaders, for us this is Dataset and DatasetGroupDIrectoryOriented, https://github.com/theislab/sfaira/blob/release/sfaira/data/dataloaders/loaders/super_group.py

See also here https://sfaira.readthedocs.io/en/latest/adding_datasets.html For modalities that cou can and want to load into h5ad, i would even recommend adopting the collection classes of sfaira, you coul just import them.

[ilibarra]

• [ ] Define generic dataloader.py using the super_group.py analogy.
• [ ] define fastq_dataloader.py and pbm_dataloader.py
• [ ] Establish YAML-based template for jolma_et_al_2013 dataset (using doi). Store in dataset
• [ ] Create more issues for other DOIs e.g. jolma_2015, PBM from Uniprobe (database), etc.