andersonwinkler / PALM

PALM: Permutation Analysis of Linear Models
70 stars 28 forks source link

Nonconformant arguments when doing one-sample t-test on CSV input data with a single DM column #25

Closed tyarkoni closed 4 years ago

tyarkoni commented 4 years ago

I'm working on a (non-imaging) Python implementation of the NPC methods in Winkler et al. (2016), and am trying to run a very simple set of analyses in PALM, just to get values I can compare my own outputs with. But I'm running into errors.

The docs suggest input data can come from a CSV file, and that the design matrix and contrast options can be omitted if one just wants a one-sample t-test (as a first pass, I'm just trying to get some output before playing with the NPC methods). This would seem to suggest that something as simple as the following should work:

palm -i sample_data.csv

...where the file contains, e.g., a single column of floats, with each row representing a different estimate.

But I get the following error:

Error using palm_partition (/Users/tal/Downloads/palm/palm_partition.m:110)
operator *: nonconformant arguments (op1 is 5x0, op2 is 5x1)

Am I missing something obvious, or is the documentation wrong? I tried feeding in design matrices, mask, and contrast files as options, but that makes no difference—it's always the same point of failure. Adding additional columns to the data CSV also doesn't help. Any suggestions would be much appreciated.

EDIT: I'm running Octave 5.2.0 in case that's relevant.

tyarkoni commented 4 years ago

Update: I got it to work by passing a design matrix with 2 columns. It fails if there's only one column in an explicitly provided DM, or if no DM is provided. I haven't looked at the code but I'm guessing in the 1-column case the DM is getting represented internally as a 1-d array instead of a 2-d array and hence breaking when the shapes don't match.

Leaving the issue open, as it does seem like a bug.

andersonwinkler commented 4 years ago

Hi Tal,

I can't reproduce the same problem with either Matlab or Octave 5.1.0. Did you get the .tar.gz from the FSL website or did you clone from Github? Please try the latter as the more recent commits may have fixed the problem (I remember vaguely this message having appeared before).

If you could, please let me know. Thanks.

All the best,

Anderson

tyarkoni commented 4 years ago

@andersonwinkler thanks for the quick reply!

I can confirm that the version in master successfully executes with a single-column DM, so it looks like that bug was indeed fixed.

That said, there does still seem to be something non-intuitive going on: if I explicitly pass a single-column DM and associated contrast, I get the expected results—i.e., a single set of outputs. But if I provide nothing but the input file (i.e., no DM or contrasts supplied), then results are produced for two contrasts. The first one is identical to the results I get from an intercept-only model when supplied explicitly; I'm not sure what the second one is.

Anyway, that might be operating as intended, but if so, it might be worth updating the docs to indicate what the output contrasts are in the event that DM and contrast files are omitted (especially the second, which I couldn't figure out). Also, in case it matters, I'm using CSVs everywhere here; I didn't check to see what happens if feeding in other file formats.

Best,

Tal

tyarkoni commented 4 years ago

Just to make it more explicit... This command:

palm -i input.csv -d design_mat.csv -t contrast.csv -npcmethod fisher -npcmod -n 500 -ise

produces these files:

 Saving file: palm_dat_tstat_uncp
 Saving file: palm_dat_tstat_fwep
 Saving file: palm_dat_npc_fisher_uncp
 Saving file: palm_dat_npc_fisher_fwep

But this command:

palm -i input.csv -npcmethod fisher -npcmod -n 500 -ise

produces these files:

 Saving file: palm_dat_tstat_uncp_c1
 Saving file: palm_dat_tstat_fwep_c1
 Saving file: palm_dat_tstat_uncp_c2
 Saving file: palm_dat_tstat_fwep_c2
 Saving file: palm_dat_npc_fisher_uncp_c1
 Saving file: palm_dat_npc_fisher_fwep_c1
 Saving file: palm_dat_npc_fisher_uncp_c2
 Saving file: palm_dat_npc_fisher_fwep_c2
andersonwinkler commented 4 years ago

Great. The version in the FSL website will be updated soon, that is, "soon"... :-) There is something I'd like to change and it's currently incompletely coded and tested.

For the 2 results: one is for the positive contrast, the other is for the negative contrast. That is:

An error will occur (not your case) if the user doesn't supply a design (it which case a column full of ones is created as design) but specifies a contrast file that is incompatible with a design that has just 1 regressor. This is an unlikely situation, though.

andersonwinkler commented 4 years ago

For the outputs: if there is just 1 output of either (a) modalities, (b) designs, or (c) contrasts, it omits the respective counter from the filename. When you specified just 1 contrast, it omitted the _c1 suffix. But when you didn't specify it, it appended _c1 and _c2.

If there were multiple designs, it would have appended to the filenames suffixes _d1, _d2, _d3, etc. Likewise for modalities: if there were multiple, it would have appended _m1, _m2, etc.

EDIT: if you always want these suffixes, even if there is just one contrast (for example, because you want a more consistent behaviour to make scripting simpler), add the option "-verbosefilenames".

tyarkoni commented 4 years ago

Thanks, that makes sense. The naming convention seems pretty clear, I just couldn't figure out why there were two contrasts. But generating + and - contrasts makes total sense now.

andersonwinkler commented 4 years ago

Perfect, thanks!