Closed balbasty closed 5 months ago
out of curiosity would it not be an issue if we one day wanted to integrate CT in BIDS: see https://bids.neuroimaging.io/bep024 ?
an issue in terms of compatibility or ease of use (where should my CT data go?)
This is a good question.
I am a bit wary of having HiP-CT as a special case of CT for several reasons:
micr/
, as far as I understandThat said, I am neither a CT nor a HiP-CT expert, let me try to get someone who knows what they are talking about to chime in.
Hi all, thought I'd jump in here to add some extra info regarding HiP-CT. @balbasty is correct for all the above. I think to summarise and add some of my own reasons to separate HiP-CT from CT: 1) HiP-CT is ex vivo only 2) It replies of propagation based phase-contrast 3) It has a hierarchical resolution format, i.e. the whole organ is imaged at an overview resolution and then specific areas can be imaged at higher resolution creating multi-scale datasets of single organ 4) As mentioed datasets are large, some of our largest aquired brain data sets are now at 14TB for one sample. 5) HiP-CT is a synchrotron technique, meaning that the meta-data fields for both acquisition and also reconstruction will be quite different and varied compared with a clinical or even a lab based CT system.
Also there is more detail on the technique in this paper , and you can see some video that give a clearer idea of the technique here.
OK it feels pretty clear from what you are saying that this is quite different from clinical CT.
From what I understand it looks like adding a microscopy suffix is needed to start supporting this.
Currently, the HiP-CT acquisition parameters are stored in the
"OtherAcquisitionParameters"
field, as a serialized JSON dictionary.
In many cases BIDS lets you add extra metadata, so no need to to have it serialized however better in this case to start listing the metadata that will be REQUIRED for downstream analysis and add those to the bids spec
@balbasty - do you want to start a PR to the spec for this?
I would not mind also seeing an example added to the bids example repo: https://github.com/bids-standard/bids-examples#contributing
looks like adding a microscopy suffix
you mean under the micr
prefix. the suffix is just hipCT
as proposed right?
seeing an example added to the bids example
perhaps a downscaled version for the moment could work. may be just at the coarsest levels. one of the the images that @balbasty linked to has 1 million objects :)
Sure I can draft a PR.
The one thing I am not sure yet is which metadata should be added (if any). @ucbpclw do you think any of the metadata should be REQUIRED or RECOMMENDED to store in the sidecar json file?
And don't worry I won't push terabytes of data to the example repo! Something like the 1mm resolution level would make sense I'd say.
I don't think we inspect the contents of zarr directories. You could drop everything but any metadata files inside that may need to be inspected to ensure consistency between OME and BIDS.
Do you mean which of metadata fields we sent in our example file are essential?
C
From: Yaël Balbastre @.> Sent: 08 November 2023 16:57 To: bids-standard/bids-specification @.> Cc: Walsh, Claire @.>; Mention @.> Subject: Re: [bids-standard/bids-specification] Add support for HiP-CT (Issue #1646)
⚠ Caution: External sender
Sure I can draft a PR.
The one thing I am not sure yet is which metadata should be added (if any). @ucbpclwhttps://github.com/ucbpclw do you think any of the metadata should be REQUIRED or RECOMMENDED to store in the sidecar json file?
And don't worry I won't push terabytes of data to the example repo! Something like the 1mm resolution level would make sense I'd say.
— Reply to this email directly, view it on GitHubhttps://github.com/bids-standard/bids-specification/issues/1646#issuecomment-1802291066, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADEWAE6FHY2WTDKDIKYSERLYDO2V3AVCNFSM6AAAAAA6WEWN66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBSGI4TCMBWGY. You are receiving this because you were mentioned.Message ID: @.***>
to illustrate what @effigies said, here are the content of the microscopy examples:
as you can see we tend to truncate large files to keep the repo light: https://github.com/bids-standard/bids-examples/blob/master/CONTRIBUTING.md#why-do-we-only-host-truncated-data-with-0kb-size
$ tree --si micr_S*
[4.1k] micr_SEM
├── [ 157] dataset_description.json
├── [ 651] participants.json
├── [ 98] participants.tsv
├── [ 168] README
├── [ 330] samples.json
├── [ 60] samples.tsv
└── [4.1k] sub-01
├── [4.1k] ses-01
│ └── [4.1k] micr
│ ├── [ 1] sub-01_ses-01_sample-A_photo.jpg
│ ├── [ 137] sub-01_ses-01_sample-A_photo.json
│ ├── [ 373] sub-01_ses-01_sample-A_SEM.json
│ └── [ 1] sub-01_ses-01_sample-A_SEM.png
├── [4.1k] ses-02
│ └── [4.1k] micr
│ ├── [ 137] sub-01_ses-02_sample-A_photo.json
│ ├── [ 1] sub-01_ses-02_sample-A_photo.tif
│ ├── [ 373] sub-01_ses-02_sample-A_SEM.json
│ └── [ 1] sub-01_ses-02_sample-A_SEM.png
├── [ 156] sub-01_sessions.json
└── [ 74] sub-01_sessions.tsv
[4.1k] micr_SEMzarr
├── [ 157] dataset_description.json
├── [ 651] participants.json
├── [ 98] participants.tsv
├── [ 210] README
├── [ 330] samples.json
├── [ 60] samples.tsv
└── [4.1k] sub-01
├── [4.1k] ses-01
│ └── [4.1k] micr
│ ├── [ 373] sub-01_ses-01_sample-A_SEM.json
│ ├── [ 1] sub-01_ses-01_sample-A_SEM.png
│ ├── [ 70] sub-01_ses-01_sample-A_SPIM.json
│ └── [4.1k] sub-01_ses-01_sample-A_SPIM.ome.zarr
│ └── [4.1k] 0
│ └── [4.1k] 0
│ └── [ 128] 0
├── [4.1k] ses-02
│ └── [4.1k] micr
│ ├── [ 373] sub-01_ses-02_sample-A_SEM.json
│ └── [ 1] sub-01_ses-02_sample-A_SEM.png
├── [ 156] sub-01_sessions.json
└── [ 74] sub-01_sessions.tsv
[4.1k] micr_SPIM
├── [ 158] dataset_description.json
├── [ 388] participants.json
├── [ 54] participants.tsv
├── [ 616] README
├── [ 330] samples.json
├── [ 83] samples.tsv
└── [4.1k] sub-01
└── [4.1k] micr
├── [ 387] sub-01_sample-A_photo.json
├── [ 1] sub-01_sample-A_photo.png
├── [ 635] sub-01_sample-A_stain-LFB_chunk-01_SPIM.json
├── [2.7k] sub-01_sample-A_stain-LFB_chunk-01_SPIM.ome.tif
├── [ 637] sub-01_sample-A_stain-LFB_chunk-02_SPIM.json
├── [2.7k] sub-01_sample-A_stain-LFB_chunk-02_SPIM.ome.tif
├── [ 636] sub-01_sample-A_stain-LFB_chunk-03_SPIM.json
├── [2.7k] sub-01_sample-A_stain-LFB_chunk-03_SPIM.ome.tif
├── [ 635] sub-01_sample-A_stain-LFB_chunk-04_SPIM.json
├── [2.7k] sub-01_sample-A_stain-LFB_chunk-04_SPIM.ome.tif
├── [ 387] sub-01_sample-B_photo.json
├── [ 1] sub-01_sample-B_photo.png
├── [ 635] sub-01_sample-B_stain-LFB_chunk-01_SPIM.json
├── [2.7k] sub-01_sample-B_stain-LFB_chunk-01_SPIM.ome.tif
├── [ 637] sub-01_sample-B_stain-LFB_chunk-02_SPIM.json
├── [2.7k] sub-01_sample-B_stain-LFB_chunk-02_SPIM.ome.tif
├── [ 635] sub-01_sample-B_stain-LFB_chunk-03_SPIM.json
├── [2.7k] sub-01_sample-B_stain-LFB_chunk-03_SPIM.ome.tif
├── [ 635] sub-01_sample-B_stain-LFB_chunk-04_SPIM.json
└── [2.7k] sub-01_sample-B_stain-LFB_chunk-04_SPIM.ome.tif
@ucbpclw
Do you mean which of metadata fields we sent in our example file are essential?
Yes, exactly
Hi
I am reopening this thread. After further discussions with "synchrotron-CT" experts (@chourroutm, @dstansby, @ucbpclw), it appeared that hipCT might not be the most adequate suffix. The "H" means hierarchical and really relates to the acqusition of multiple images in the same sample. Each individual image is a (very fancy) phase-contrast CT.
They have suggested "PC" for phase-contrast, although in the current spec, "PC" relates to optical phase-contrast, not xray phase-contrast.
It might be good to fix this suffix before the next releast of the spec.
@satra @ayendiki @effigies @yarikoptic
XPC - X-ray Phase Contrast?
Adding the X does make it sound cooler.
From: Chris Markiewicz @.> Sent: Friday, April 26, 2024 7:53 AM To: bids-standard/bids-specification @.> Cc: ayendiki @.>; Mention @.> Subject: Re: [bids-standard/bids-specification] Add support for HiP-CT (Issue #1646)
External Email - Use Caution
XPC - X-ray Phase Contrast?
— Reply to this email directly, view it on GitHubhttps://secure-web.cisco.com/1RJ4VJxNzwdOx__6K5a5aSKoHEBWWcoRNu31bi88izNWrHHlE3pEACHrvA_Imz4Pkx1iLs-17wyg7mQvMFnJWhnLtvMqlO41J4qyb4pkww2b_4FTUR0H9rG14DE4L4JpbPXj26a7I6zAohXT7LwH_OB3GjIc9VCcTsX2c-s3GXOW2nIADNpeBGa3dEWYGrzkut9SxcD3TCXPTPzTZSqkQbRx723V9_nLpGsqNIU6WMmz7tFfqyVv_J4DH-QqDwwCBj3xTXzmNHbTRy4pg2BSCkL-Xiu7MJy5-RTOiUFYVZ8vbrBnyqH8EBAwOuGkC4R1sfupd-z0-utxIOivmv7eMpQ/https%3A%2F%2Fgithub.com%2Fbids-standard%2Fbids-specification%2Fissues%2F1646%23issuecomment-2079237706, or unsubscribehttps://secure-web.cisco.com/1oGFgqJFwUAtlKAwMqRdsiqqCI3yVq4UA5x61nHovlUGc0-Kg19k0ttBr4m13I2Ox1ck8HvWzRv-vopF23Ij0-Ne9WFr9hFYbDXzSPsyL_wExaY0_l9pyZubBJFltOqdTJp1V-hOn4wsL8xq_rWdaTMqoJr3Dh0hprk7CxcZ1CZNd9juHILZV_xFy8mDlxU_mU_PJDnwBSVOaKSlQOY947ZXM-860g0tO_4aw-m6-l6lKKiTan9p-3fhJ__llZGvpxIvNkNECNhDEeY_p0hlHAZ9sEJU2FSZaqWQt620w07GIbfP7T6wIOxM3Z7RVW9AWJpBy0_ymvfOdoixmwiOAzg/https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADU34VYLIELDK3ILVD5V3ETY7I52XAVCNFSM6AAAAAA6WEWN66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZZGIZTONZQGY. You are receiving this because you were mentioned.Message ID: @.***>
[ { @.": "http://schema.org", @.": "EmailMessage", "potentialAction": { @.": "ViewAction", "target": "https://github.com/bids-standard/bids-specification/issues/1646#issuecomment-2079237706", "url": "https://github.com/bids-standard/bids-specification/issues/1646#issuecomment-2079237706", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { @.": "Organization", "name": "GitHub", "url": "https://github.com" } } ] The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://www.massgeneralbrigham.org/complianceline https://www.massgeneralbrigham.org/complianceline . Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
Agree that it's worth dropping the "Hi" of "HiP-CT" - as mentioned above stands for Hierarchical which is about how mutiple datasets/scans relate to each other (basically some are higher resolution VOIs of others).
XPC sounds good. Does the suffix need to indicate it's CT data though? I note there's already a microCT suffix in the spec. So how about something like xPCCT? Or just PCCT for phase contrast computed tomography?
XPC sounds good. Does the suffix need to indicate it's CT data though?
The only hard and fast rule is that it needs to be alphanumeric. Otherwise, if the affected community agrees that it's precise enough, and people outside the community don't see it conflicting with something familiar to them, then it's all good.
I note there's already a microCT suffix in the spec. So how about something like xPCCT? Or just PCCT for phase contrast computed tomography?
I don't love xPCCT, but XPC and PCCT seem equally fine to me. But I don't work with these data, so I'm happy to leave it to those who do.
If there's no particular standard/rules for what has to be in the suffix, apart from understandable by the community, then maybe it's worth keeping this as hipCT?
@dstansby I'd rather go for a broader name, as we're the only group/consortium doing HiP-CT (at the moment) but the community using the same underlying technique --- with or without the hierarchical component, which indeed rather corresponds to a dataset rather than a sole volume --- is already quite developed .
The different names I am aware of are:
As there is no real consensus of the naming within the community, we will not please everyone, of course. It seems to me that XPC might be sufficient as a suffix to the microscopy
keyword. Otherwise, to convey the 3D, maybe XPCT is the one to go.
What's the opinion of the others? @ucbpclw @balbasty @satra My former collaborators were also interested in such an addition, I can include them in the decision-making.
Then, what is the course of events? Should I draft a PR with one of the two suffices?
I suppose we can prepare in the following months (once the validator recognizes the suffix) a truncated version of the open brain dataset from the Human Organ Atlas: https://human-organ-atlas.esrf.eu/datasets/572252538
@chourroutm - i would leave this to you and the other experts to decide. as long as it doesn't conflict, we should be ok with any choice of a new extension.
@chourroutm I vote XPCT, but yes it'd be great to have a validation from other groups, so that it lives up to the test of time :)
Once it's ok'd, you (or I) should revert the changes from PR #1686 and replace all occurences of hipCT by XPCT.
It would be good to get an example in https://github.com/bids-standard/bids-examples as well, so that we can update and test the validator. See the contributing guide.
I would vote XPCT as well, it is the wording we are generally using when writing grants as it captures all the essential components of the technique and is specific enough to differentiate it from other X-ray or tomography examples.
@gobej and @DoctorEmmetBrown do you want to weigh in?
@effigies I've tailored an example dataset in chourroutm/bids-xpct-example/tree/master/micr_XPCTzarr that we'll amend before I draft a PR to bids-standard/bids-examples.
@ucbpclw @dstansby and eventually @balbasty can I have your inputs on the dataset? This is the file structure of the example dataset:
micr_XPCTzarr
| dataset_description.json
| participants.json
| participants.tsv
| README.md
| samples.json
| samples.tsv
|
+---code
| data_retrieval.py
| data_retrieval_reqs.txt
| metadata_formatter.py
| metadata_formatter_reqs.txt
| truncate_data.sh
|
\---sub-LADAF-2020-31
| sub-LADAF-2020-31_sessions.json
| sub-LADAF-2020-31_sessions.tsv
|
\---ses-01
\---micr
| sub-LADAF-2020-31_ses-01_sample-brain_photo.jpg
| sub-LADAF-2020-31_ses-01_sample-brain_photo.json
| sub-LADAF-2020-31_ses-01_sample-brain_XPCT.json
|
\---sub-LADAF-2020-31_ses-01_sample-brain_XPCT.ome.zarr
| .zattrs
| .zgroup
|
+---0
| | .zarray
| [...]
[...]
|
+---4
|
[...]
|
\---9
+---0
| 0
| 1
| 2
|
+---1
| 0
| 1
| 2
|
\---2
0
1
2
Layout looks good to me (FWIW I sucessfully adapted a HiP-CT dataset to BIDS to upload to LINC), but I think that's beyond the scope of this issue (which is just about the prefix)?
I am also in favour of "XPCT".
XPCT adopted in #1822.
Dear BIDS community
Hierarchical Phase-Contrast Tomography (HiP-CT) is an ex vivo X-ray technique that leverages high-energy X-ray sources -- specifically, the ESRF's Extremely Brilliant Source (EBS). HiP-CT is non-destructive, reaches resolutions of 1 to 10 um, and allows specific ROIs to be imaged within the sample without physical sectioning. Although the modality is obviously not widely available, the UCL group has a relatively streamlined acquisition process and we will be collecting data on multiple brains in the coming years.
We would like to add support for HiP-CT in BIDS, and we think it most naturally fits as a new modality suffix within the microscopy extension. This makes sense because micro-CT is already supported as a microscopy suffix. Also, the microscopy extension already includes tags that are required by HiP-CT data (for example, the existence of multiple images of the same samples, potentially with different resolutions and different field-of-views, labeled as
chunk-
). We are therefore proposing to add thehipCT
suffix to the microscopy extension.We have uploaded an exampled dataset that follows the proposed convention to DANDI: https://dandiarchive.org/dandiset/000026/draft/files?location=sub-I58%2Fses-Hip-CT%2Fmicr&page=1 We also provide three neuroglancer links that allow each volume to be visualized online, directly streaming the dandi data:
Let me also include a typical metadata file provided by the UCL group:
And the corresponding BIDS json file
Currently, the HiP-CT acquisition parameters are stored in the
"OtherAcquisitionParameters"
field, as a serialized JSON dictionary. But it may be better to move some of these HiP-CT-specific tags into the BIDS spec, and we would welcome any feedback from users of the other microscopy modalities.@satra @Kevancic @dstansby
Thanks Yael