Next steps for BIDS-Prov

[effigies]

I'm opening this issue to figure out what a finalized BIDS-Prov would look like. A couple options:

1) A PR to the specification (like bids-standard/bids-specification#487), and BIDS-Prov is 100% maintained within BIDS. 2) A separate repository for the format specification, like BIDS-Stats Models (https://bids-standard.github.io/stats-models/) or BIDS Execution (https://bids-standard.github.io/execution-spec/), with a relatively confined PR to BIDS proper that makes the relationship clear.

Other arrangements could be imagined, but I think the "in BIDS" or "alongside BIDS" distinction is the main one to settle on, and then work out the details.

Other questions:

1) How much, if any, validation should be done by the BIDS validator? Will there be a 3rd-party validator to call out to, as with HED? 2) For existing tooling in this repository, would there be a plan to merge it into another library, or keep it as its own thing? Will it be distributed on PyPI?

@bids-standard/steering @bids-standard/maintainers

[robertoostenveld]

In terms of documenting the format specification of BIDS-Prov within the BIDS specification, I can imagine that it would be shortly mentioned in the https://bids-specification.readthedocs.io/en/stable/02-common-principles.html or in the https://bids-specification.readthedocs.io/en/stable/03-modality-agnostic-files.html sections, and then elaborated upon in an appendix.

I can imagine that the BIDS-Prov specific tooling (like the visualizer, but possibly also a non-graphical validator) could continue to live in their own repository under the bids-standard organization.

[sappelhoff]

If future BIDS extensions like derivatives for ephys heavily depend on BIDS-Prov (i.e., to specify ephys derivatives one MUST use BIDS-Prov), then I think BIDS-Prov should be maintained 100% within BIDS.

Else, if BIDS-Prov becomes something like a "recommended way to document provenance" but not strictly necessary within BIDS, then I would vote for it living in its own repository (like statsmodels, execution, ... ) with only a short mention in the BIDS spec.

---

• re: validator --> I think irrespective of how the above decision turns out, maintaining a separate provenance validator that is "just integrated" into the BIDS validator would be simpler (I may be wrong with this assumption), and with the HED validator we already have a precedent

• re: General prov tooling --> I agree with Robert in https://github.com/bids-standard/BEP028_BIDSprov/issues/125#issuecomment-1739485848 --> they should live in their own repos under the bids-standard organization

[effigies]

If future BIDS extensions like derivatives for ephys heavily depend on BIDS-Prov (i.e., to specify ephys derivatives one MUST use BIDS-Prov), then I think BIDS-Prov should be maintained 100% within BIDS.

I think we could have a situation where the BIDS-Prov format definition lives on its own, but some way of defining required entries could still live in BIDS. Similar to how we specify JSON metadata keys/values without defining JSON inside BIDS.

[CPernet]

I do not think any derivatives should rely heavily/depend on BIDS-Prov (jsonld) which is why we have introduced the descriptions.tsv, allowing to specify all the steps performed, but if one wants to include a full reproducible workflow then use BIDS-Prov

From a more general perspective, BIDS is about documenting data, and BIDS-Prov documents a process (which I agree gives rise to the data, still it's not the same as a json next to a file). In that respect it does make sense to live next to the main spec.

[cmaumet]

I am inviting @bclenet to the discussion as he will be working on BIDS-Prov form now.