search

bids-standard / bids-specification

Brain Imaging Data Structure (BIDS) Specification
https://bids-specification.readthedocs.io/
Creative Commons Attribution 4.0 International
275 stars 157 forks source link

Which BIDSVersion? #545

Closed geowk closed 4 years ago

geowk commented 4 years ago

Hello,

We are starting to convert are datasets to BIDS format.

In the dataset_description.json file, the BIDSVersion is listed.

"BIDSVersion | REQUIRED. The version of the BIDS standard that was used."

How does one determine what BIDSVersion is used? Is it based on a specific conversion pipeline (e.g, dcm2bids ,etc.). We convert our data from dicom format to nifiti using dcm2niix, but don't see any BIDSversion listed in the .json files. We've arranged the data in BIDS format based on the tutorial (http://reproducibility.stanford.edu/bids-tutorial-series-part-1a/#man3) using a bash script and then use the bids_validator to confirm the dataset.

Thanks.

g.

Remi-Gau commented 4 years ago

Good question (and by that I mean I have no clear idea how to answer it). :smile:

Tried to see into the archives of issues of the validator if there is anything about this.

The is issue mentions this:

"The validator should validate the latest released version of the standard."

So I don't know if the validator checks that the version mentioned in the dataset description matches the requirement of that specific version.

But this highlights the fact that the validator could do 2 things that could help many users:

sappelhoff commented 4 years ago

So I don't know if the validator checks that the version mentioned in the dataset description matches the requirement of that specific version

It does not - and I am not sure whether that's a good or bad thing. Ideally of course, the validator would check BIDS datasets according to the BIDSVersion field --> and a dataset formatted under BIDS 1.2 would pass the validator under 1.2 requirements, but potentially fail under 1.4 requirements.

But such a logic is not built into the validator. That piece of software can only validate one "BIDS version" (the newest stable release, 1.4 currently).

have an output message mentioning the BIDS version it is using

that sounds good to me in principle, yet practically the validator does not have full coverage of the BIDS specification (yet) and additionally, it is lagging behind the specification in terms of development. Given these two points, it would currently be a lie of the validator to claim to be validating according to version 1.4.

To JavaScript knowledgeable BIDS contributors: If you read this, please head over to the "low hanging fruits" project --> we would very much welcome some developers over at bids-validator!

How does one determine what BIDSVersion is used?

IMHO it's up to the conversion / curating software to tell you according to which standard version the processing is happening. For example in mne-bids we print the BIDSVersion that we use as an output of our pipeline automatically.

And if you are manually curating your BIDS dataset, you would put into this field the version of the docs that you used to prepare your dataset. Ideally you would always use the newest stable release (1.4 currently)

geowk commented 4 years ago

Thank you both for your feedback.

It appears that people using various pipelines to format the data into BIDS might run into this issue.

For example, take the tutorials listed in this site:

https://github.com/bids-standard/bids-starter-kit/wiki/Tutorials

Both of these walk-through examples were very straightforward and helpful, yet they were created in 2018 and reference BIDSVersions 1.0.2. and 1.1.0, respectively.

If the current stable release is 1.4, that means thing would have changed.

Should BIDS users only use the following website with the pdf link as the template:

https://bids.neuroimaging.io/specification.html https://bids-specification.readthedocs.io/en/stable/

How often are newer BIDSVersions released, and what is the most efficient way for users to know when new versions are released?

Regarding the bids-validator, it is stated the latest release of bids-validator always look for the datasets adhering to the latest BIDSVersion format. As long as we keep bids-validator up to date, we are assured that it will determine if your dataset meets the latest BIDSVersion criteria?

Thanks again.

g.

sappelhoff commented 4 years ago

It appears that people using various pipelines to format the data into BIDS might run into this issue.

yes, I can imagine that.

How often are newer BIDSVersions released,

we don't have a stable period or releases, but looking at the releases page can perhaps give you a feeling for it. We are following semver and increment a minor version whenever we add a new BIDS extension proposal, and we increment a patch version whenever we feel like some threshold is reached of new clarifications or inclusions and we don't want to wait until the next minor increment.

The major increment is reserved for breaking changes that'll come about with BIDS 2.0 --> no timeline exists for that as of yet.

sappelhoff commented 4 years ago

hat is the most efficient way for users to know when new versions are released?

subscribing to our channels

As long as we keep bids-validator up to date, we are assured that it will determine if your dataset meets the latest BIDSVersion criteria?

yes, keeping your bids-validator up to date is a good idea. If it passes at timepoint 1, it's highly likely that it will also pass at timepoint 2 in the future, because in BIDS we are keeping breaking changes to an absolute minimum until BIDS 2.0 comes around.

That means the only way your BIDS formatted dataset that passes the validator now can fail in the future is for these reasons:

  1. a breaking change was made because a serious bug was discovered and absolutely HAD to be fixed (very low probability, happened about 2 times since 2016, but I don't now exact numbers right now)
  2. the bids-validator coverage has increased and it now catches bugs that were bugs already before ... but they simply went unnoticed
yarikoptic commented 4 years ago

FWIW: Re validating against a specific bids version, eventually #475 would open possibility to validator(s) to validate at least parts of the specification against the specified BIDS version.

geowk commented 4 years ago

thanks sapplehhoff for the clarification

g