Post-hoc masking with nifti images does not appear to do anything

jacob-lee commented 5 years ago

::SwE v.1.2.0 ::SPM12

In Classic parametric SwE, explicit masking with a nifti image works if included in the SwE design, e.g.

matlabbatch{1}.swe.smodel.masking.em = fullfile(masks_dir, 'gray_matter_mask.nii')

However, post-hoc masking using the Results dialogue appears to do nothing.

Analysis

The mask is never applied because the variable Im having the mask's path information is overwritten to [] before the masks are applied.

In swe_getSPM.m

Mask = spm_input('apply masking','+1','b','none|contrast|image',[0,1,2],1);

solicits a mask (line 435). Choosing an image file sets Im to a cellstr with the image's path.

[pf,nf,ef] = spm_fileparts(Im{1}) splits the mask's path to get mask name (line 527), but this appears to only be used to construct a label: pf, nf and ef are not otherwise used.

Then beginning on line 556, the variable holding the path to the mask Im is set to the empty matrix.

Therefore it is empty when, beginning on line 664, each mask applied in the for loop:

for i = 1:numel(Im)
...
...
end

Commenting out lines 556-558 resolves my issue. I suspect doing so might cause an issue with wild bootstrapped analyses, given the preceding comment in the code.

BryanGuillaume commented 5 years ago

@nicholst, I am currently working on this issue and I believe that I fix the bug reported. However, looking at the code, I am a bit suprise that the post-hoc masking is entirely disabled for WB analyses. Is there a particular reason for this? To my understanding, the post-hoc masking in SPM (and in SwE) simply masks some voxels in the display but without modifying any scores or p-values. Thus, I guess that could also be used for the WB. Am I correct about this?

nicholst commented 5 years ago

I can’t imagine why it is disabled. I don’t recall it as an action. @TomMaullin, do you have any memory of it?

TomMaullin commented 5 years ago

Hi @nicholst @BryanGuillaume , I must admit I can't remember why this is the case... I had thought we implemented post-hoc masking for WB at some point but maybe we never actually did... Perhaps either we didn't ever get around to it or we decided against it because the mask is important in the clustersize bootstrap?

If we had implemented it my guess would be that we implemented some form of post-hoc masking under the small volume corrections option in the results menu for both WB and parametric... the PR that would have been associated with was this one.

BryanGuillaume commented 5 years ago

Reading @TomMaullin's comment, I think a potential reason for not having post-hoc masking for WB would be that the masking of clusterwise WB p-value images could be weird as the clusters may be reduced in size. However, I do not think that this is wrong per se provided that the user understand well what the masked image actually represents.

Another reason would be that, with the WB, we cannot mask using a contrast like for a paramatric analysis because we do not have access to other contrasts directly within the current WB analysis. Nevertheless, we can still use the option to use an external image as a mask (like implemented for the parametric analyses).

@nicholst, does SnPM actually implements post-hoc masking?

nicholst commented 5 years ago

Oh wait! I remember now!

The issue is the FWE corrections... for FWE voxel- cluster- & TFCE inference the corrections depend on the search space, and so if you use a post-how mask, the corrected p-values will no longer be right... the post-how mask, which is smaller, would imply more significant FWE p-values as you have a smaller search region.

So! As I see it, we have two choices: We either (a) leave it as is, and better document why there is no post-hoc masking (just like in SnPM), (b) we add post-hoc masking, but in that case remove / set to NaN all FWE corrected p-values.

What do you two think? --

BryanGuillaume commented 5 years ago

The thing is that I do not think that the post-hoc masking in SPM (and consequently in SwE) reduces the search space before computing the p-values or the threshold. I believe that it reduces the search space after computing the p-values and the threshold. Thus, that should not be an issue for the WB.

As a proof of my statement above,

1) Here below, the help text about masking found in spm_getSPM and swe_getSPM

% Masking simply eliminates voxels from the current contrast if they
% do not survive an uncorrected p value (based on height) in one or
% more further contrasts.  No account is taken of this masking in the
% statistical inference pertaining to the masked contrast.

2) If we look how the FDR threshold is computed, we have in the code

u = spm_uc_FDR(u,Inf,'Z',n,VspmSv,0); where VspmSv = cat(1,xCon(Ic).Vspm);.

As xCon(Ic).Vspm are the list of unmasked score images, that seems to indicate that the masking does not influence at all the FDR threshold.

Thus, in this case, the post-hoc masking can be used for any type of inference even for FWE corrected p-values. The only thing that is annoying is that the user needs to read swe_getSPM to understand what the masking is actually doing (see point 1 above). This problem seems the same in SPM.

nicholst commented 5 years ago

You’re right! There is then, for me, only one question: Do SPM cluster p-values change on the basis of post-hoc masking... I think they do... i.e. while the voxel-wise FWE corrected p-values may not change (for a voxel that is still in the post-hoc mask, the size of clusters will change and so the cluster FWE p-values will change.

I’m fine with this behavior, but I just want to be sure that we’re matching SPM’s behavior. --

BryanGuillaume commented 5 years ago

Yes @nicholst, you are right! In SPM, while all the p-values are indeed computed based on the unmasked image (i.e. do not account for the reduced research space from the masking), when a cluster becomes smaller due to the post-hoc masking, its cluster p-values are based on its new reduced size. And the voxel-wise p-values are unchanged. Please see the 2 results below confirming this behaviour:

SPM Before Post-hoc masking

SPM After Post-hoc masking

I am not sure I like this behaviour, particularly because this does not seem well documented, but that is how SPM works. Thus, it might be better to mimic this as much as possible. And the good news is that it is relatively easy to do for the clusterwise WB as the bootstraped maximum cluster sizes are actually saved in SwE.mat. Thus we can easily recompute the p-values of the clusters in the masked image based on these max cluster sizes obtained with the unmasked image. That seems to match SPM's behaviour.

If it is alright for you, I will then implement this behaviour for the WB.

nicholst commented 5 years ago

Thanks for checking Bryan!

Yes, for better or worse, let’s mimic the SPM behavior (& it’s easy).

BryanGuillaume commented 5 years ago

I believe the initial issue (i.e. Post-hoc masking for parametric analysis) has been resolved in PR #125 (see the 2 example images below). Thus I will close this issue and add the post-hoc masking of WB analysis as a separate "enhancement" issue.

Parametric analysis before post-hoc masking

Parametric analysis after post-hoc masking

NISOx-BDI / SwE-toolbox