search

NISOx-BDI / SwE-toolbox

SwE toolbox
GNU General Public License v2.0
16 stars 7 forks source link

Rationalise filenames #9

Closed nicholst closed 6 years ago

nicholst commented 6 years ago

Hi @BryanGuillaume ,

As the use of SwE is growing, with @TomMaullin, we've been looking at doing several updates to SwE to improve usability.

This is the beginning of a discussion on how to rationalise the filenames of output across different modes of the program. Ideally I'd like to follow Anderson's general approach that he uses in PALM-like naming system; see PALM's Output files documentation. However, that really only covers statistics and p-values and not everything else (frankly, I'm happy to keep with the covvis* convention you use, but we do need to tidy up the various different ways of representing the original test statistics (T,F) with varying DF, and the converted Z,X images.

I had hoped to offer a concrete proposal but I am out of time right now and have instead just produced an inventory of the various filenames.

Let me know if you have time/energy/interest to work with us on this. No worries if not

-Tom

An adaptation of PALM's conventions for our statistics and p-value outputs

File name File contents
swe_{unit}_{stat}_c{#}.{ext} Statistic for contrast c{#}.
swe_{unit}_{stat}_{pval}_c{#}.{ext} P-value for contrast c{#}.

Where we change to using a swe prefix to avoid confusion with SPM output, and...

Field Description
{unit} The "unit" of type of inference; as with PALM, this can be vox for voxelwise, vtx for vertexwise, dat for any data that doesn't necessarily have a spatial meaning (e.g., data entered in mat format), clustere for cluster extent (allowing for, some day, clusterm for cluster mass support, and tfce for TFCE).
{stat} The name of the statistic, which can be Tstat for the t-statistic, Fstat for the F-statistic, Zstat for the Z-statistic, Xstat for the chi-squared-statistic; if the statistic is based on an inversion (probability integral transform) of Wild Bootstrap p-values, then WB is appended.
{pval} The type of p-value stored in the image. It can be uncp for uncorrected permutation p-values, FWEp for FWER-corrected p-values, FDRp for FDR-adjusted p-values
c{#} Contrast index, in the same order as in the files supplied with the -t and/or -f options.
{ext} File extension. This will typically be the same extension as the corresponding input files.

I thought carefully about it, and I like Anderson's use of a letter c prefix on the contrast; if we take on this convention we can also apply it to our covariance images, e.g. having them become, e.g. cov_beta_b001_b003 or cov_beta_g002_b001_b003 or cov_vis_g002_v003_v005.

In what follows I have made no particular proposals for any changes, but simply have tried to record the "facts on the ground" of what is produced at present.

Parmametric, NIFTI

File name Description
mask Mask
beta_{b#} Regression coefficent number {b#}
ResI_{y#} Residual for observation {y#} (later deleted)
con_{c#} GLM contrast {c#}
cov_beta_{b1#}_{b2#} Covariance between regression coefficient {b1#} and {b2#}
cov_beta_g_{g#}_{b1#}_{b2#} Covariance between regression coefficient {b1#} and {b2#} for group {g#}
cov_vis_{g#}_{v1#}_{v2#} Covariance between errors, visit {v1#} and {v2#} for group {g#}
edf_{c#} Effective degrees of freedom for contrast {c#}
spmUncP_{c#} Uncorrected P-values for contrast {c#}
spm{S}_{c#} Test statistics for contrast {c#}, where {S} is one of T or F (original "raw" statistics) or Z or X (transformed based on EDF to Z or chi2_1)

Parmametric, mat file output

File name Description
mask.mat Mask
beta.mat Regression coefficents
cov_beta.mat Covariance between regression coefficients
cov_vis.mat Covariance between errors, visits x visits
cov_beta_g.mat Covariance between errors, visits x visits by group
spm?_{c#}.mat Test statistics for contrast {c#}, where {S} is one of T or F (variable score saved) or Z or X (transformed based on EDF to Z or chi2_1; variable equivalentScore saved)
spmUncP_{#}.mat Uncorrected P-values for contrast {c#} (variable uncP saved)
edf_{#}.mat Effective degrees of freedom for contrast {c#} (variable edf saved)

Wild Bootstrap, NIFTI

File name Description
mask Mask
score Test statistic, approximately a T or F statistic
ResWB_{y#} Null hypothesis residuals for observation {y#}
YfittedWB_{y#} Null hypothesis model fitted values for observation {y#}
lP{+-} -log10 uncorrected voxelwise p-values from WB, where {+-} is one of + or -
lP_FWE{+-} -log10 FWE-corrected voxelwise p-values from WB, where {+-} is one of + or -
lP_FDR{+-} -log10 FDR-corrected voxelwise p-values from WB, where {+-} is one of + or -
lP_clusterFWE{+-} -log10 FWE-corrected clusterwise p-values from WB, where {+-} is one of + or -

Wild Bootrap, mat file output

File name Description
mask.mat Mask
beta.mat Regression coefficients
score.mat Test statistic, approximately a T or F statistic
{mn}Score.mat WB null distribution of extremal score statistic, where {mn} is one of min or max
maxClusterSize{N}.mat WB null distribution of maximal cluster size, where {N} is one of ` (empty) orNeg`
{l}uncP_{pn}.mat WB uncorrected voxelwise p-values, where {pn} is one of pos or neg, {l} is one of ` (empty) orl` (-log10)
{l}fwerP_{pn}.mat WB FWE-corrected voxelwise p-values (based on score statistic), where {pn} is one of pos or neg, {l} is one of ` (empty) orl` (-log10)
{l}fdrP_{pn}.mat WB FDR-corrected voxelwise p-values (based on WB uncorrected p-values), where {pn} is one of pos or neg, {l} is one of ` (empty) orl` (-log10)
{l}clusterFwerP_{pn}_perElement.mat WB FWE-corrected clusterwise p-values (based on EDF-converted score) by each element, where {pn} is one of pos or neg, {l} is one of ` (empty) orl` (-log10)
{l}clusterFwerP_{pn}_perCluster.mat WB FWE-corrected clusterwise p-values (based on EDF-converted score) by cluster, where {pn} is one of pos or neg, {l} is one of ` (empty) orl` (-log10)

CS test, NIFTI

File name Description
CS_test_F CS test F
CS_test_p CS test p
nicholst commented 6 years ago

In the next comment I'm going to have a go at updating the file names... hopefully this will mostly reflect what @TomMaullin has done, but I realise that the table in the comment above is perfectly suited to review our naming decisions in one single place.

TomMaullin commented 6 years ago

Hi @nicholst ,

I don't know if this helps but I have also uploaded a list of updated filenames in my google drive to help demonstrate that the updates had not affected the output of the toolbox. (See https://drive.google.com/drive/folders/1Y33znbq2ob1XADV5dUxN_xp7386V1JZ5?usp=sharing ).

nicholst commented 6 years ago

Parmametric, NIFTI

File name Description
mask swe_vox_mask
beta_{b#} swe_vox_beta_b{b#}
ResI_{y#} swe_vox_resid_y{y#}
con_{c#} swe_vox_{T\|F}con_c{c#}
cov_beta_{b1#}_{b2#} swe_vox_cov_b{b1#}_b{b2#}
cov_beta_g_{g#}_{b1#}_{b2#} swe_vox_cov_g{g#}_b{b1#}_b{b2#}
cov_vis_{g#}_{v1#}_{v2#} swe_vox_cov_g{g#}_v{v1#}_v{v2#}
edf_{c#} swe_vox_edf_c{c#}
spmUncP_{c#} swe_vox_{T\|F}stat_lp_c{c#}
spm?_{c#} swe_vox_{T\|F\|zT\|xF}stat_c{c#}
con_{c#} swe_vox_beta_c{c#}

Parmametric, mat file output

File name Description
mask.mat swe_vox_mask
beta.mat swe_vox_beta_bb
cov_beta.mat swe_vox_cov
cov_vis.mat swe_vox_cov_vv
cov_beta_g.mat swe_vox_cov_g_bb
spm?_{c#}.mat swe_vox_{T\|F\|Z\|X}stat_c{c#}
spmUncP_{#}.mat swe_vox_{T\|F}stat_lp_c{c#}
edf_{#}.mat swe_vox_edf_c{c#}
con_{#}.mat swe_vox_beta_c{c#}

Wild Bootstrap, NIFTI

File name Description
mask swe_vox_mask
score swe_vox_{T\|F}stat_c{c#}
lP{+-} swe_vox_{T\|F}stat_lp-WB_c{c#} contrast 1,2 replaces +,-
lP_FWE{+-} swe_vox_{T\|F}stat_lpFWE-WB_c{c#}
lP_FDR{+-} swe_vox_{T\|F}stat_lpFDR-WB_c{c#}
lP_clusterFWE{+-} swe_clustere_{T\|F}stat_lpFWE-WB_c{c#}
ResWB_{y#} swe_vox_resid_y{y#}
YfittedWB_{y#} swe_vox_fit_y{y#}

Wild Bootstrap, mat file output

File name Description
mask.mat swe_vox_mask
beta.mat swe_vox_beta_bb
score.mat swe_vox_{T\|F\|zT\|xF}stat_c{c#} contrast 1,2 replaces +,-
{mn}Score.mat swe_vox_{T\|F}stat_maxperm_c{c#}
{l}uncP_{pn}.mat swe_vox_{T\|F}stat_lp-WB_c{c#}
{l}fwerP_{pn}.mat swe_vox_{T\|F}stat_lpFWE-WB_c{c#}
{l}fdrP_{pn}.mat swe_vox_{T\|F}stat_lpFDR-WB_c{c#}
{l}clusterFwerP_{pn}_perElement.mat swe_clustere_{T\|F}stat_lpFWE-WB_c{c#}
ResWB_{y#}.mat swe_vox_resid_y{y#}
YfittedWB_{y#}.mat swe_vox_fit_y{y#}

CS test, NIFTI

File name Description
CS_test_F swe_vox_Fstat-Box
CS_test_p swe_vox_Fstat-Box_lp
TomMaullin commented 6 years ago

Hi @nicholst

Apologies I was under the impression the filenames were already in this format for the parametric/non-bootstrap SwE and have only implemented these changes for the output maps in the case when a wild bootstrap has been run. If you would like I can also do this for the parametric case in this PR? This should not take too long as this will be a case of performing changes to swe_cp.m nearly identical to the changes I have applied to swe_cp_WB.m.

nicholst commented 6 years ago

I don't want to get distracted... I will review the PR leaving your name changes in place... will do a separate PR to possibly revise names later.

nicholst commented 6 years ago

OK... the previous comment is my current thinking on the best filename structure... most important revelation is that "pos" "neg" (or actually "" and "neg") really just need to be handled in the framework of contrast numbers. In fact, we should anticipate that we will eventually support multiple contrasts.

But I will now returning to reviewing #8 without implementing these... want to merge that first before we do this.

nicholst commented 6 years ago

Thoughts on the number of significant digits: 4 dp's for, e.g. contrasts, when the software doesn't even allow more than 1 right now seems silly!

Counter Initial Format
Scan y %04d
Beta b %02d
Group g %02d
Visit v %02d
Contrast c %02d
nicholst commented 6 years ago

After #19, only one thing left to do on this issue… update the documentation to match the new files written with these conventions.

TomMaullin commented 6 years ago

The documentation has now been updated to match #19 . I've closed this issue as it has now been addressed by PR #19.

nicholst commented 6 years ago

Hi @TomMaullin - One tiny fix to docs is needed: See https://github.com/NISOx-BDI/SwE-toolbox/blob/master/swe_getSPM.m#L218 where lP_FWE+.img is referenced (change it to a reference to a swe_vox_<blah> image while avoiding a specific extension. Also the next few lines there reference the {neg} usage which is no longer relevant.

Also in that file, search for and correct the double ii in iinformation.

TomMaullin commented 6 years ago

Oh I see! Apologies I thought you meant the file name change documentation on this comment thread! Yes, I'll change this.

nicholst commented 6 years ago

Well, a separate issue is that we’ll need to migrate this documentation above to a proper location, like on NISOx.org/Software/SwE ...

TomMaullin commented 6 years ago

This discussion has now been fully recorded by providing the documentation on the NISOx website here. As the issue has been resolved there is no need to keep this thread open.