"Saving p-values" never finishes with GNU Octave

[tobac]

This issue seems to have been reported before (eg #10) but as it is closed I decided to open a new one.

I'm trying to run the following PALM analysis with GNU Octave on HCP subjects:

palm -i data_sub.nii -d design/design.mat -t design/design.con -o results_dense_subcortical -n 500 -corrcon -logp -accel tail -nouncorrected -T

palm -i data_L.func.gii -d design/design.mat -t design/design.con -o results_dense_cortical_L -T -tfce2D -s L_midthickness.surf.gii L_area.func.gii -logp -corrcon -nouncorrected -accel tail -n 500

palm -i data_R.func.gii -d design/design.mat -t design/design.con -o results_dense_cortical_R -T -tfce2D -s R_midthickness.surf.gii R_area.func.gii -logp -corrcon -nouncorrected -accel tail -n 500

I have used the exact same commands with Octave successfully before, but when adding more subjects to my analysis (80 vs. 247), Octave seems to get stuck in a loop or something. The last line to be output is the following:

Saving p-values (uncorrected, and corrected within modality and within contrast).

After that, nothing happens for days (!). Switching to MATLAB solves this problem (saving takes ~ 3 s). I'd rather use Octave, though.

The same thing happened to me before with a rather large number of study subjects.

[andersonwinkler]

Hi, Thanks for letting me know. Which Octave version are you using and which operating system? Also, may I ask if you could send me your files (input, design and contrast files)? You could place in some file-sharing service and send the link via email. Thanks. Anderson

[tobac]

GNU Octave, version 4.0.3 on Debian, i.e. the version provided in the Debian stable repository at the moment.

Email is on its way.

[tobac]

Update: This issue might indeed be fixed in more recent Octave versions. I set up an Arch Linux VM on our server in order to use the latest stable version et voilà: it works.

Working setup: Running PALM alpha115 using Octave 5.1.0

[andersonwinkler]

Excellent!

[tobac]

I'm afraid I'm facing the same issue with regard to another analysis I'm trying to run. This time, both Octave and MATLAB won't finish saving.

I'd be happy to send you the input files.

[tobac]

When using MATLAB, trying to kill the process will output the function and line number at which the script hangs (presumably), so this might be helpful:

Computing p-values.
Saving p-values (uncorrected, and corrected within modality and within contrast).
Operation terminated by user during palm_pareto (line 116)

In palm_saveall (line 246)
                        Ptosave =
                        palm_pareto(plm.G{y}{m}{c},plm.Gmax{y}{m}{c},false,opts.accel.tail_thr,opts.accel.G1out);

In palm_core (line 2489)
palm_saveall(plm,opts);

In palm (line 81)
palm_core(varargin{:});

[andersonwinkler]

Hi @tobac,

Yes, if you could send the input files I'd be happy to have a look. Use any file sharing service then send me the link. I'm reluctant to paste my email here because of bots; you may be able to find it easily, though (e.g., in the FSL mailing list, or in the Yale or NIH directories).

All the best,

Anderson

[tobac]

Hi @andersonwinkler,

did you get the email I sent to your Gmail address two days ago?

[andersonwinkler]

I did! But I couldn't reply yet (I can no longer keep up with emails, some always fall through the cracks...; please give me a few days).

In any case, the message indicates the process was killed by the user. Could it be that you hit some wall time or memory limits?

[ashlea-segal]

I am experiencing the same problem with PALM alpha116 MATLAB r2019b. I am trying to run the following PALM analysis on HCP subjects as well:

palm -i data_sub.nii -o results_dense_subcortical -n 10000 -corrcon -logp -accel tail -nouncorrected -T

It seems to get stuck at the same point @tobac mentioned above:

Saving p-values (uncorrected, and corrected within modality and within contrast).

Did you find a solution to this issue?

Thank you in advance, Ashlea

[andersonwinkler]

Hi Ashlea,

I'm trying to locate the email exchange with @tobac but I don't seem to find any resolution to this. May I suggest that you try the current version on Github and let me know if the problem persists? Also try with a different seed (e.g., -seed 632), and try also with the gamma approximation as that tends to be quite as good in general.

Thanks!

All the best,

Anderson

On Sun, 8 Nov 2020 at 04:10, ashlea-segal notifications@github.com wrote:

I am experiencing the same problem with PALM alpha116 MATLAB r2019b. I am trying to run the following PALM analysis on HCP subjects as well:

palm -i data_sub.nii -o results_dense_subcortical -n 10000 -corrcon -logp -accel tail -nouncorrected -T

It seems to get stuck at the same point @tobac https://github.com/tobac mentioned above:

Saving p-values (uncorrected, and corrected within modality and within contrast).

Did you find a solution to this issue?

Thank you in advance, Ashlea

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/andersonwinkler/PALM/issues/18#issuecomment-723549649, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIGUNE4FOXSTP5OMOD2XNLSOZOBTANCNFSM4HEGTHDQ .

[ashlea-segal]

Thank you for your prompt reply, @andersonwinkler!

I tried the current version on Github and a different seed, neither worked, however the gamma approximation ran but the fwep results were NaNs. Following your suggesting on the HCP mailing list (here), I included the -precision double flag, both approximation methods worked. The results however, vary in significance. Using the gamma approximation, there is a range of significant values (1-6), whereas using the tail approximation, a high majority of significant values are 4 (logp).

[andersonwinkler]

Hi Ashlea,

Great, nice to hear the -precision flag solved the issue. Sorry it didn't occur to me that you could be entering a single-precision .nii file.

About choosing between gamma and tail: experimentally we found that the two are remarkably similar, but the tail has a stronger theoretical support for FWER as it uses a generalized Pareto distribution (that approximates the maximum of any set of random variables) fitted only to the tail (the part of the distribution we care about).

If the two differ, I would go with tail, though gamma isn't usually too off. The many ties you are observing (4, which is -log10(1/10000)) may mean that these voxels have a strong signal and more permutations may be needed for a good tail fit. Ties aren't a problem, though -- it's ok to have them in permutation tests.

All the best,

Anderson

On Mon, 9 Nov 2020 at 23:58, ashlea-segal notifications@github.com wrote:

Thank you for your prompt reply, @andersonwinkler https://github.com/andersonwinkler!

I tried the current version on Github and a different seed, neither worked, however the gamma approximation ran but the fwep results were NaNs. Following your suggesting on the HCP mailing list (here https://www.mail-archive.com/hcp-users@humanconnectome.org/msg06595.html), I included the -precision double flag, both approximation methods worked. The results however, vary in significance. Using the gamma approximation, there is a range of significant values (1-6), whereas using the tail approximation, a high majority of significant values are 4 (logp).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/andersonwinkler/PALM/issues/18#issuecomment-724451474, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIGUNBYREVXYJLOPBVIRC3SPDB7LANCNFSM4HEGTHDQ .

[ashlea-segal]

Great, thank you so much for your assistance!

Thanks,

Ashlea

On 10 Nov 2020, at 11:47 pm, A. M. Winkler notifications@github.com wrote:

Hi Ashlea,

Great, nice to hear the -precision flag solved the issue. Sorry it didn't occur to me that you could be entering a single-precision .nii file.

About choosing between gamma and tail: experimentally we found that the two are remarkably similar, but the tail has a stronger theoretical support for FWER as it uses a generalized Pareto distribution (that approximates the maximum of any set of random variables) fitted only to the tail (the part of the distribution we care about).

If the two differ, I would go with tail, though gamma isn't usually too off. The many ties you are observing (4, which is -log10(1/10000)) may mean that these voxels have a strong signal and more permutations may be needed for a good tail fit. Ties aren't a problem, though -- it's ok to have them in permutation tests.

All the best,

Anderson

On Mon, 9 Nov 2020 at 23:58, ashlea-segal notifications@github.com wrote:

Thank you for your prompt reply, @andersonwinkler https://github.com/andersonwinkler!

I tried the current version on Github and a different seed, neither worked, however the gamma approximation ran but the fwep results were NaNs. Following your suggesting on the HCP mailing list (here https://www.mail-archive.com/hcp-users@humanconnectome.org/msg06595.html), I included the -precision double flag, both approximation methods worked. The results however, vary in significance. Using the gamma approximation, there is a range of significant values (1-6), whereas using the tail approximation, a high majority of significant values are 4 (logp).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/andersonwinkler/PALM/issues/18#issuecomment-724451474, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIGUNBYREVXYJLOPBVIRC3SPDB7LANCNFSM4HEGTHDQ .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/andersonwinkler/PALM/issues/18#issuecomment-724680238, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALKY2LYOB5YO7L6R74TM5KTSPEY7BANCNFSM4HEGTHDQ.

[JAQuent]

I am using pal, Jun/2021 (version alpha119) and Matlab both on a computing cluster as well as locally on my computer running Ubuntu and I have the same issue (stuck at "Computing p-values.") when attempting a resting-state connectivity analysis on HCP-style data.

palm -i Y.dscalar.nii -transposedata -d design_matrix.csv -t contrast.csv -ise -o "results_${ROI_NAME}" -n 5000 -corrcon -logp -accel tail -fdr -zstat -ise -precision double

Adding the -precision flag does seem to solve the issue but I will keep you updated if it crops up again. Interestingly, four other (larger) ROIs don't have this issue only the two smallest. Is it possible that the two smaller ROIs that I use create a problem because the correlation with itself is so high? Would it make sense to exclude these voxels/vertices?