BEP016: Forced data representation specification

[Lestropie]

Thought that came to me this morning regarding the distinction between what are currently referred to as "model fit parameters" and "model-derived parameters".

Consider models that provide a combination of isotropic and anisotropic data. Examples might include:

• FSL bedpostx ball-and-sticks, which has both discrete fibre orientations in spherical coordinates and voxel-wise scalar values such as mean & standard deviation of diffusivity.

• MRtrix3 MSMT CSD, which provides multiple spherical harmonics images. Under typical usage there is one such image with lmax > 0 (ie. anisotropic) and multiple other images with lmax = 0 (ie. isotropic). However the latter of these are not scalar images; they are still spherical harmonic images, which therefore include a sqrt(4pi) scaling due to the spherical harmonic basis.

One might therefore consider the prospect that, for model fit parameters, it should be compulsory that metadata field "OrientationRepresentation" be specified. This is in part because in NIfTI it is not possible to distinguish between an image with dimensions X x Y x Z and one with dimensions X x Y x Z x 1; the latter would be reasonably interpreted as an SH image whereas the former is not. Regardless, forcing this field to be present would provide an unambiguous distinction between these two cases, which would be self-contained and not dependent on the contents of any other files.

Now consider instead a model-derived parameter such as FA. Obviously this is a scalar image. However making it compulsory to specify ""OrientationRepresentation": "scalar"" in the sidecar of such an image would seem an unusual demand. So permitting the inference of being a scalar representation in the case of a 3D image with no OrientationRepresentation field would, to me, be more reasonable in the model-derived parameter case than it would for the model fit parameter case.

This PR is flagged as draft:

• It's moreso a thought experiment rather than something I'm suggesting should absolutely be adopted.
• It shows a scenario where distinguishing between model fit parameter and model-derived parameter has more of a consequence than just defining different suffices, but there is some chance of that distinction dissolving away.

[codecov[bot]]

Codecov Report

Merging #67 (74e0cbb) into bep-016 (ed53944) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff            @@
##           bep-016      #67   +/-   ##
========================================
  Coverage    88.53%   88.53%           
========================================
  Files            6        6           
  Lines         1055     1055           
========================================
  Hits           934      934           
  Misses         121      121           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[arokem]

This is interesting. I'm wondering whether there aren't cases of things that are not model-fit parameters that also need the "OrientationRepresentation". For example, a representation of peaks of an ODF detected using a peak-finding algorithm and stored in an X x Y x Z x N x 3 file would probably need this field, even though it's not an mfp.

[Lestropie]

There absolutely are: fibre orientations could be an mfp in the case of ball-and-sticks, or an mdp in the case pf spherical deconvolution. This is wholly supported by the BEP as it currently stands.

The specific distinction I'm contemplating here is the case of simple scalar images: in the case of an mdp, it would be most convenient to not be forced to specify ""OrientationRepresentation": "scalar"" and just have that inferred from the fact that it's a 3D image, whereas for mfp, it would be safer to not make such an inference, and instead demand that ""OrientationRepresentation"" be provided to distinguish between ""scalar"" and ""sh"".

[arokem]

Thanks for the clarification. Do I understand correctly that this only matters for cases of things that could have a singleton last dimension then? Is it really just the lmax=0 sh images? The ball and stick examples don't seem to fit this.

[Lestropie]

I can't really figure out exactly what the question is; there's too many "the"'s and "this"'s and "it"'s. But I'll opine a bit more and see if anything helps it to stick.

For anything that does contain orientation information, that metadata field will always be compulsory. For any 4D image with more than one volume, it is impossible to determine from the image size alone what type of orientation data is encoded. For instance, if you have an image with six volumes, it could be:

• 3 spherical coordinates;
• 2 spherical coordinates with an amplitude for each;
• 2 3-vectors;
• SH with lmax=2;
• Tensor coefficients. Even in the least ambiguous case, with 2 volumes, you would still want to know whether it's a single spherical coordinate or a vector of parameters of unknown nature.

If you now reduce to the 3D image case, there's still a potential ambiguity: it could be a scalar image, or it could be an SH image with l=0 (and the limitations of NIfTI precludes us from telling the difference through image dimensionality). Therefore, the safest approach is to demand that the nature of the data representation always be explicitly defined.

But if someone just wants to take a tensor fit and generate an FA image, it may be difficult to explain to them that they have to explicitly state that that image is a scalar map, as opposed to something else. It's far from intuitive as to why that would be necessary.

My point here is: If you wanted to relax the restriction on having to always explicitly state that data representation for every image, one way in which you might define the scenario in which you permit its absence would be:

1. The image must have no more than three non-singleton axes.

   This is just about the only case where there is a reasonable "default" data representation that can be inferred without any explicit definition of such.

2. The image must be a model-derived parameter rather than a model fit parameter.

   Because there are different model fit parameters from different models that result in three non-singleton axes, but some of them are scalars whereas some of them are SH, it would not IMO be reasonable to infer a "default" in the case where no data representation is explicitly provided.

The reason this is relevant for the discussion during the BIDS Connectivity workshop is that point 2 provides a plausible scenario under which having a suffix-based distinction between model fit parameters and model-derived parameters is of utility.

[arokem]

OK. What I understand so far:

The "OrientationRepresentation " metadata field will be required for any nifti file that contains a 4D volume (because you can't store 4D data with singleton last-dimension). It will also, in addition, be required for any model-fit parameter that is stored in a 3D nifti, because that nifti could contain the lmax=0 sh coefficient in each voxel.

Is that correct? Another approach would be to assume that 3D model parameters (whether fit or derived) is scalar unless otherwise specified in the metadata. Would that fail for any of your use-cases? Maybe that's easier to explain?

[Lestropie]

Correct and correct.

Again, I'm not actually saying that this is something I want to adopt. For instance, one could follow your second suggestion, and say that anything with three non-singleton dimensions is a scalar unless one explicitly sets ""OrientationRepresentation": "sh"", and this would be the case for both MFP and MDP (regardless of whether or not those two things continue to be distinguished). This was more of a thought experiment exploring whether or not it would introduce a use case where there was merit in persisting with the distinction between MFP and MDP. It was never going to be a nail in the coffin, and after contemplating I don't think it's particularly convincing. I'll see how I go coming up with other scenarios.

[arokem]

Yes -- I am increasingly convinced that a distinction between model-fit parameters and model-derived parameters would not buy us much and is often mired in ambiguity.

[Lestropie]

I think the comment above may be intended for a different issue thread; likely #69.

I would note that as per my last comment in #68, if the _micro suffix were to be adopted for DWI models (and would apply to both fit and derived parameters), that would resolve well with this PR, in that anything with the _micro suffix would be REQUIRED to specify "OrientationRepresentation".

[arokem]

My comment certainly relates to #69, but it was more of a reply to this specific sentence:

This was more of a thought experiment exploring whether or not it would introduce a use case where there was merit in persisting with the distinction between MFP and MDP.

I continue to be skeptical that this distinction will prove useful -- this specific issue being a case in point. And yes, we can continue discussing in #68. The _micro suffix is an interesting idea.

[Lestropie]

I continue to be skeptical that this distinction will prove useful

FYI I'm leaning in this direction myself. In my own head I find it an important distinction, but in the context of a discrete BIDS App generating a finite set of derivatives, ie. the user is not responsible for the progression of fitting a model and then deriving parameters from the model, it largely loses its function. If this is the best exemplar justification I can some up with for it, it's probably a losing argument.

If this distinction is removed, the discussion in this PR remains relevant in that we must decide if there is any circumstance in which, for anything with the relevant suffix (eg. "_micro"), field "OrientationRepresentation" can be omitted, or whether it should be compulsory across the board. The use case is images with only 3 non-unity axes, and whether it should be compulsory to specify "OrientationRepresentation" = "scalar" in order to disambiguate from "OrientationRepresentation" = "sh", or whether the former should be assumed in the case of the absence of the latter.

[arokem]

This argument makes sense to me:

Now consider instead a model-derived parameter such as FA. Obviously this is a scalar image. However making it compulsory to specify ""OrientationRepresentation": "scalar"" in the sidecar of such an image would seem an unusual demand. So permitting the inference of being a scalar representation in the case of a 3D image with no OrientationRepresentation field would, to me, be more reasonable in the model-derived parameter case than it would for the model fit parameter case.

So, I think the rule should be that a file with a _micro suffix (or whatever suffix we end up with here) is not required to have the "OrientationRepresentation" field in its metadata unless it is either 4D (or more) or if it is 3D has "OrientationRepresentation" other than "scalar". If it is 3D and does not have an "OrientationRepresentation" field, it is assumed to be "scalar" (but this can also be specified explicitly, if so desired). Would that be too tricky to validate?

[arokem]

As an aside, should we already start using macros for the tables? Might be a topic for a separate issue.

[Lestropie]

Would that be too tricky to validate?

From a conceptual viewpoint, not at all. But I have precisely zero knowledge of the internal operation of the validator to know what is or is not difficult to validate. In this particular scenario, the validator would need to be able to read NIfTI headers and attribute dimensionality based on the axis sizes (since NIfTI doesn't store dimensionality explicitly, it relies on interpreting trailing axes of unary size). No idea if it reads NIfTI headers at all elsewhere.

[Lestropie]

Closed due to being superseded by #92.