support for microscopy

[Remi-Gau]

Was testing how pybids behaves with the recent microscopy extension and it does not seem to be able to index them.

See this notebook: https://github.com/Remi-Gau/pybids/blob/remi-bids_example/examples/bids_example.ipynb

The other "new" modalities (PET, qMRI, ASL) seem OK. So not sure what is happening.

Edit The other modality seem OK (like PET) provided one updates the config, but microscopy seems to fail still.

related note: to support more data types, any plans to update the default_path_patterns or use the schema to generate them?

[mariehbourget]

Thanks @Remi-Gau for opening this issue! I was also wondering how to integrate new data types to pybids, is it with the schema or separate?

In our lab, we use pybids to index files for MRI, M/EEG and Microscopy datasets.

If that helps, here is what we have been doing during the development of the Microscopy BEP:

• We use "force_index" in "BIDSLayoutIndexer" to index files in sub-XX folders based on data type (micr) and microscopy file extensions. "force_index" allows us to keep the validator on for MRI and M/EEG.
• We use "force index" to index the new samples.tsv and samples.json files.
• We use a "custom" config file in "BIDSLayout" that includes the new "sample", "stain" and "chunk" entities as well as the "micr" data type.

It would be great to integrate Microscopy in pybids so we can skip all of that. Also, IIUC pybids can be used for other query (ex: path construction), which we do not cover with our workaround.

[thomasbeaudry]

@Remi-Gau Hi, so when you say "The other "new" modalities (PET, qMRI, ASL) seem OK." do you mean that pybids 0.14 supports PET (and ASL) and can detect it. I'm scanning a PET bids folder, and when i try get_datatype() it's returning null. I would like to know if I should expect it to find the PET files

[Remi-Gau]

@thomasbeaudry

Updated my example with what pybids (0.14.0) does on one of the bids-example PET dataset.

https://github.com/Remi-Gau/pybids/blob/remi-bids_example/examples/bids_example.ipynb

It seems to be able to find pet data.

[thomasbeaudry]

@Remi-Gau Yes, your example seems to work. My test bids directory of 1 subject doesn't though:

bids_layout.get() `[,

, , , , , ]` bids_layout.get_datatype() ['anat']

Would you see why it wouldn't? Thanks!

[effigies]

We need to update the datatype regex here:

https://github.com/bids-standard/pybids/blob/f70eb2fce7e601d110c3b1788f6c837c28177644/bids/layout/config/bids.json#L89-L90

Feel free to submit a quick PR. And we should also add new entities introduced by recent BEPs to the config.

[thomasbeaudry]

@effigies Hey editing the pattern to include pet returns the datatype for my test directory! :)

[Remi-Gau]

Oops. Sorry. Had not checked my config: I had indeed modified it in a previous commit with a message that very not explicit...

{
            "name": "datatype",
            "pattern": "[/\\\\]+(anat|beh|dwi|eeg|fmap|func|ieeg|meg|perf|pet|micr)[/\\\\]+"
},

See commit here: https://github.com/Remi-Gau/pybids/commit/18a593b5ee3ac5136744d6c2b9555c5a30628abc

Will update the top message to avoid further misunderstanding.

@effigies Should I make a tiny PR to update the config?

[mariehbourget]

Hey @Remi-Gau , If that helps, here is the custom config file that we use in our lab for indexing Microscopy with the new entities "sample", "stain" and "chunk" and the data type "micr": https://github.com/ivadomed/ivadomed/blob/master/ivadomed/config/config_bids.json

[Remi-Gau]

Thanks @mariehbourget

I think I will try to use this to make a PR to update the config generally.

[thomasbeaudry]

@mariehbourget I looked at your config and I noticed that you have CT defined. What does your bids structure look like for CT? thanks!

[mariehbourget]

Hi @thomasbeaudry,

I had forgotten about that!

We needed to use a "bids-like" structure for CT for one of our project but CT is not in BIDS. However there is an extension proposal for it (BEP024). We based our structure on the template in the BEP024 document. So we temporarily used the data type "ct" as in the BEP template and added it to our custom config file.

(@Remi-Gau, you can ignore "ct" in my config as it is not in BIDS)