an error for ICA_FIX, don't find file of fixversion

[RuiWang0805]

cat: /home/qwbrain/HCPpipelines/ICAFIX/fixversion: No such file or directory
/home/qwbrain/HCPpipelines/ICAFIX/hcp_fix_multi_run: line 423: cd: /home/qwbrain/data/Pipelines_ExampleData/100307/MNINonLinear/Results/tfMRI_WM_RL: No such file or directory
Sun Sep 29 18:01:02 CST 2024:hcp_fix_multi_run: While running '/home/qwbrain/HCPpipelines/ICAFIX/hcp_fix_multi_run --fmri-names=/home/qwbrain/data/Pipelines_ExampleData/100307/MNINonLinear/Results/tfMRI_WM_RL/tfMRI_WM_RL@/home/qwbrain/data/Pipelines_ExampleData/100307/MNINonLinear/Results/tfMRI_WM_LR/tfMRI_WM_LR@/home/qwbrain/data/Pipelines_ExampleData/100307/MNINonLinear/Results/tfMRI_GAMBLING_RL/tfMRI_GAMBLING_RL@/home/qwbrain/data/Pipelines_ExampleData/100307/MNINonLinear/Results/tfMRI_GAMBLING_LR/tfMRI_GAMBLING_LR@/home/qwbrain/data/Pipelines_ExampleData/100307/MNINonLinear/Results/tfMRI_MOTOR_RL/tfMRI_MOTOR_RL@/home/qwbrain/data/Pipelines_ExampleData/100307/MNINonLinear/Results/tfMRI_MOTOR_LR/tfMRI_MOTOR_LR --high-pass=0 --concat-fmri-name=/home/qwbrain/data/Pipelines_ExampleData/100307/MNINonLinear/Results/tfMRI_WM_GAMBLING_MOTOR_RL_LR/tfMRI_WM_GAMBLING_MOTOR_RL_LR --motion-regression=FALSE --training-file=HCP_Style_Single_Multirun_Dedrift.RData --fix-threshold=10 --delete-intermediates=FALSE --config= --processing-mode=HCPStyleData':
Sun Sep 29 18:01:02 CST 2024:hcp_fix_multi_run: ERROR: 'cd' command failed with return code: 1

===> ERROR: Command returned with nonzero exit code
---------------------------------------------------
         script: hcp_fix_multi_run
stopped at line: 423
           call: cd `dirname $fmri`
       hostname: ubuntu
      exit code: 1
---------------------------------------------------

===> Aborting execution!

[glasserm]

Please post how you called MR+FIX.

[RuiWang0805]

Please post how you called MR+FIX.

@glasserm Thanks for your reply, sorry, I don't understand your intention

[glasserm]

What was the commandline you used to invoke hcp_fix_multi_run.

[RuiWang0805]

@glasserm ./hcp_fix_multi_run.sh

[glasserm]

Did you run fMRIVolume and fMRISurface yet? Have you installed ICAFIX?

[RuiWang0805]

![Uploading ICA_FIX_dir.png…]()

What was the commandline you used to invoke hcp_fix_multi_run. @glasserm I have run fMRIVolume and fMRISurface and git clone ICAFIX

[glasserm]

What does this show in the terminal: ls /home/qwbrain/data/Pipelines_ExampleData/100307/MNINonLinear/Results/tfMRI_WM_RL

[RuiWang0805]

What does this show in the terminal: ls /home/qwbrain/data/Pipelines_ExampleData/100307/MNINonLinear/Results/tfMRI_WM_RL I only processed the 100307-fMRI raw data and did't processed the tfMRI data.

[RuiWang0805]

What does this show in the terminal: ls /home/qwbrain/data/Pipelines_ExampleData/100307/MNINonLinear/Results/tfMRI_WM_RL @glasserm I processed the resting state fMRI data I collected myself, not the HCP data.

[glasserm]

Then you need to point the HCP pipelines to your own data, not that of 100307.

[RuiWang0805]

Then you need to point the HCP pipelines to your own data, not that of 100307. Sorry bother you again, I run it and produced many files including the following: I'm not sure if it has run successfully. 2.txt hcp_fix.e5536.txt hcp_fix.o5536.txt

[glasserm]

Those logs state that you haven't set up compiled matlab properly or set it to use interpreted matlab. We will soon also support pyfix with the HCP Pipelines, and alternative to the R and matlab-based FIX.

[RuiWang0805]

Thanks for your reply! I run the ICAFIX with the way of MCR and set /HCP pipeline/ICAFX/setting.sh as following: I've tried to run it many times and I can't find anything wrong with it. Look forward to your help. Thank you settings.txt

[glasserm]

"Unable to find MATLAB Compiler Runtime" suggests you don't have the MCR set up on your path or correctly in the settings file.

[RuiWang0805]

"Unable to find MATLAB Compiler Runtime" suggests you don't have the MCR set up on your path or correctly in the settings file. Thank you for your reply. I have many questions to bother you again @glasserm:

1. My MATLAB is R2016a, set.sh (line 3th)FIXFEATUREVERSION What number should I fill in?
2. I want to use compiled MATLAB,FSL_FIX_MCRROOT Specifies whether to contain the MCR version information, such as /usr/local/matlab_Runtime /v9.1, or only the upper-layer directory, such as /usr/local/matlab_matlab_Runtime?
3. HCPpipeline/Example/SettingUP.sh (line 24th), export MATLAB_COMPILER_RUNTIME=' ' should I set MCR default installation path, for example: / usr/local/MATLAB/MATLAB_Runtime, or the installation position of MCR package, for example: / home/qwbrain/HCPpipelines/MCR？

[coalsont]

"Unable to find MATLAB Compiler Runtime" suggests you don't have the MCR set up on your path or correctly in the settings file. Thank you for your reply. I have many questions to bother you again @glasserm:

1. My MATLAB is R2016a,  set.sh (line 3th)FIXFEATUREVERSION What number should I fill in?

I don't know what set.sh is. You should not edit the FIXFEATUREVERSION in FIX's settings.sh, that needs to stay 1.06 or I think it will break when it loads the training data.

2. I want to use compiled MATLAB,FSL_FIX_MCRROOT Specifies whether to contain the MCR version information, such as /usr/local/matlab_Runtime /v9.1, or only the upper-layer directory, such as /usr/local/matlab_matlab_Runtime?

I think the most recent matlab fix is compiled against v93. The folder name on the end should look like this:

https://github.com/Washington-University/HCPpipelines/blob/f41f7294f22be191f6fa44011c865199160fdff0/Examples/Scripts/SetUpHCPPipeline.sh#L24

3. HCPpipeline/Example/SettingUP.sh (line 24th), export MATLAB_COMPILER_RUNTIME=' ' should I set MCR default installation path, for example: / usr/local/MATLAB/MATLAB_Runtime, or the installation position of MCR package, for example: / home/qwbrain/HCPpipelines/MCR？

Other pipeline scripts need to use this value to locate the MCR, so it should point to wherever it is actually installed in the environment that you will launch the script in. I don't know if the fix-related pipelines specifically need to use this setting, though. We compile against v93, so you would need that version (unless you are recompiling).

[RuiWang0805]

"Unable to find MATLAB Compiler Runtime" suggests you don't have the MCR set up on your path or correctly in the settings file. Thank you for your reply. I have many questions to bother you again @glasserm:

1. My MATLAB is R2016a,  set.sh (line 3th)FIXFEATUREVERSION What number should I fill in?

I don't know what set.sh is. You should not edit the FIXFEATUREVERSION in FIX's settings.sh, that needs to stay 1.06 or I think it will break when it loads the training data.

2. I want to use compiled MATLAB,FSL_FIX_MCRROOT Specifies whether to contain the MCR version information, such as /usr/local/matlab_Runtime /v9.1, or only the upper-layer directory, such as /usr/local/matlab_matlab_Runtime?

I think the most recent matlab fix is compiled against v93. The folder name on the end should look like this:

https://github.com/Washington-University/HCPpipelines/blob/f41f7294f22be191f6fa44011c865199160fdff0/Examples/Scripts/SetUpHCPPipeline.sh#L24

3. HCPpipeline/Example/SettingUP.sh (line 24th), export MATLAB_COMPILER_RUNTIME=' ' should I set MCR default installation path, for example: / usr/local/MATLAB/MATLAB_Runtime, or the installation position of MCR package, for example: / home/qwbrain/HCPpipelines/MCR？

Other pipeline scripts need to use this value to locate the MCR, so it should point to wherever it is actually installed in the environment that you will launch the script in. I don't know if the fix-related pipelines specifically need to use this setting, though. We compile against v93, so you would need that version (unless you are recompiling).

Thanks for your reply! My MCR is v901, that means I should download and install v93, MATLAB2017b？

[RuiWang0805]

"Unable to find MATLAB Compiler Runtime" suggests you don't have the MCR set up on your path or correctly in the settings file. Thank you for your reply. I have many questions to bother you again @glasserm:

1. My MATLAB is R2016a,  set.sh (line 3th)FIXFEATUREVERSION What number should I fill in?

I don't know what set.sh is. You should not edit the FIXFEATUREVERSION in FIX's settings.sh, that needs to stay 1.06 or I think it will break when it loads the training data.

2. I want to use compiled MATLAB,FSL_FIX_MCRROOT Specifies whether to contain the MCR version information, such as /usr/local/matlab_Runtime /v9.1, or only the upper-layer directory, such as /usr/local/matlab_matlab_Runtime?

I think the most recent matlab fix is compiled against v93. The folder name on the end should look like this: https://github.com/Washington-University/HCPpipelines/blob/f41f7294f22be191f6fa44011c865199160fdff0/Examples/Scripts/SetUpHCPPipeline.sh#L24

3. HCPpipeline/Example/SettingUP.sh (line 24th), export MATLAB_COMPILER_RUNTIME=' ' should I set MCR default installation path, for example: / usr/local/MATLAB/MATLAB_Runtime, or the installation position of MCR package, for example: / home/qwbrain/HCPpipelines/MCR？

Other pipeline scripts need to use this value to locate the MCR, so it should point to wherever it is actually installed in the environment that you will launch the script in. I don't know if the fix-related pipelines specifically need to use this setting, though. We compile against v93, so you would need that version (unless you are recompiling).

Thanks for your reply! My MCR is v901, that means I should download and install v93, MATLAB2017b？ I suspect that ICAFIX has higher requirements for the version, such as the version of Ubuntu, MCR, and MATLAB.

[RuiWang0805]

"Unable to find MATLAB Compiler Runtime" suggests you don't have the MCR set up on your path or correctly in the settings file. Thank you for your reply. I have many questions to bother you again @glasserm:

1. My MATLAB is R2016a,  set.sh (line 3th)FIXFEATUREVERSION What number should I fill in?

I don't know what set.sh is. You should not edit the FIXFEATUREVERSION in FIX's settings.sh, that needs to stay 1.06 or I think it will break when it loads the training data.

2. I want to use compiled MATLAB,FSL_FIX_MCRROOT Specifies whether to contain the MCR version information, such as /usr/local/matlab_Runtime /v9.1, or only the upper-layer directory, such as /usr/local/matlab_matlab_Runtime?

I think the most recent matlab fix is compiled against v93. The folder name on the end should look like this: https://github.com/Washington-University/HCPpipelines/blob/f41f7294f22be191f6fa44011c865199160fdff0/Examples/Scripts/SetUpHCPPipeline.sh#L24

3. HCPpipeline/Example/SettingUP.sh (line 24th), export MATLAB_COMPILER_RUNTIME=' ' should I set MCR default installation path, for example: / usr/local/MATLAB/MATLAB_Runtime, or the installation position of MCR package, for example: / home/qwbrain/HCPpipelines/MCR？

Other pipeline scripts need to use this value to locate the MCR, so it should point to wherever it is actually installed in the environment that you will launch the script in. I don't know if the fix-related pipelines specifically need to use this setting, though. We compile against v93, so you would need that version (unless you are recompiling).

Thanks for your reply! My MCR is v901, that means I should download and install v93, MATLAB2017b？

[RuiWang0805]

@glasserm @coalsont I suspect that ICAFIX has higher requirements for the version, such as the version of Ubuntu, MCR, and MATLAB.

[coalsont]

There is no requirement on OS version, as long as FSL and freesurfer will run on it. The purpose of compiled matlab mode is so that you only need an MCR installed (free download), rather than having Matlab installed in your processing environment (licensing cost, periodic renewal steps or license manager setup required). You need to download the MCR version that exactly matches the compilation version, which is 9.3 for everything inside our pipelines, and should also be 9.3 for the matlab-based FSL FIX.

[RuiWang0805]

There is no requirement on OS version, as long as FSL and freesurfer will run on it. The purpose of compiled matlab mode is so that you only need an MCR installed (free download), rather than having Matlab installed in your processing environment (licensing cost, periodic renewal steps or license manager setup required). You need to download the MCR version that exactly matches the compilation version, which is 9.3 for everything inside our pipelines, and should also be 9.3 for the matlab-based FSL FIX.

Thanks for your reply! I wii try to install MATLAB R2017b and run ICAFIX.

[RuiWang0805]

I installed MATLAB R2017b and run ICAFIX, The log with the final error message is empty(files:hcp_fix.e7658,hcp_fix.e8477), The hcp fix.o7658 run log displays the following information: processing FMRI file rfMRI_REST1_LR with highpass 0 running MELODIC running FIX FIX Feature extraction for Melodic output directory: rfMRI_REST1_LR.ica create edge masks run FAST registration of standard space masks extract features No valid training file specified No valid labelling file specified Could not find a supported file with prefix "rfMRI_REST1_LR.ica/filtered_func_data_clean"

[RuiWang0805]

There is no requirement on OS version, as long as FSL and freesurfer will run on it. The purpose of compiled matlab mode is so that you only need an MCR installed (free download), rather than having Matlab installed in your processing environment (licensing cost, periodic renewal steps or license manager setup required). You need to download the MCR version that exactly matches the compilation version, which is 9.3 for everything inside our pipelines, and should also be 9.3 for the matlab-based FSL FIX.

@coalsont Sorry to bother you again. Looking forward to your reply. Thank! I installed MATLAB R2017b and run ICAFIX, The log with the final error message is empty(files:hcp_fix.e7658,hcp_fix.e8477), The hcp fix.o7658 run log displays the following information:

processing FMRI file rfMRI_REST1_LR with highpass 0 running MELODIC running FIX FIX Feature extraction for Melodic output directory: rfMRI_REST1_LR.ica create edge masks run FAST registration of standard space masks extract features No valid training file specified No valid labelling file specified Could not find a supported file with prefix "rfMRI_REST1_LR.ica/filtered_func_data_clean"

The hcp_fix.o8477 run log displays the following information:**

No valid 4D_FMRI input file specified

I Not sure if this has run successfully, I took a subject we collected to run the ICAFIX noise reduction process, so no training samples

[glasserm]

Look in the R logfile for the R package errors.

[coalsont]

What version of the pipelines are you using? Both hcp_fix and hcp_fix_multi_run should print out the locations of these extra log files (to the .o<number> file) before trying to run fix. Note, on each run, a new number is used for the .o<number> and .e<number> files, you should only be looking at the output from your most recent attempt.

Do ls -a rfMRI_REST1_LR.ica, it should show .fix.log and .fix_2b_predict.log, those are the extra log files to look at. I don't know why FSL hid them by putting . as the first character of the filename. There is also rfMRI_REST1_LR.ica/fix/logMatlab.txt to look at.

[RuiWang0805]

Thanks for your reply! @coalsont i. What version of the pipelines are you using? The fix version is 1.06.15 and the version of MATLAB_Runtime is v93. The version of MATLAB is R2017b. ii. I usually run the IcaFixProcessingBatch.sh to finish noise reduction process. iii. The rfMRI_REST1_LR/ RFMRI_rest1_lr.ica folder contains the following contents (fig.ls rfMRI_REST1_LR/ RFMRI_rest1_lR.ica)

iv. The .fix.log includings four files (show in the attachment hcp_fix.o7658, hcp_fix.e7658, hcp_fix.o8477, hcp_fix.e8477). hcp_fix.o7658.txt hcp_fix.o8477.txt The hcp_fix.e7658 and the hcp_fix.e8477 is empty. v. The logMatlab.txt is shown in the attachment. logMatlab.txt

[coalsont]

What does running $HCPPIPEDIR/show_version say? Or qunex --version if you are running things that way.

[RuiWang0805]

What does running $HCPPIPEDIR/show_version say? Or qunex --version if you are running things that way. The version of HCPpipeline is Post-v4.7.0-MOD-9cfa37a.

[coalsont]

Please do cd $HCPPIPEDIR; git status. The MODIFIED: YES line means you may have changed something you weren't supposed to.

The error message that our version of hcp_fix gives says "Invalid 4D_FMRI" and gives the name of the fMRI it couldn't find. With that clue I found:

https://git.fmrib.ox.ac.uk/fsl/fix/-/blob/master/hcp_fix#L38

You did not run the HCP Pipelines version of the hcp_fix script, you have run the one included with the FSL FIX download, which is not up to date. Use this launcher script and edit its variables, instead of composing your own hcp_fix command:

https://github.com/Washington-University/HCPpipelines/blob/master/Examples/Scripts/IcaFixProcessingBatch.sh

[RuiWang0805]

Please do cd $HCPPIPEDIR; git status. The MODIFIED: YES line means you may have changed something you weren't supposed to.

The error message that our version of hcp_fix gives says "Invalid 4D_FMRI" and gives the name of the fMRI it couldn't find. With that clue I found:

https://git.fmrib.ox.ac.uk/fsl/fix/-/blob/master/hcp_fix#L38

You did not run the HCP Pipelines version of the hcp_fix script, you have run the one included with the FSL FIX download, which is not up to date. Use this launcher script and edit its variables, instead of composing your own hcp_fix command:

https://github.com/Washington-University/HCPpipelines/blob/master/Examples/Scripts/IcaFixProcessingBatch.sh

Sorry, I usually run the IcaFixProcessingBatch.sh to finish noise reduction process and occur log of hcp_fix

[RuiWang0805]

Please do cd $HCPPIPEDIR; git status. The MODIFIED: YES line means you may have changed something you weren't supposed to. The error message that our version of hcp_fix gives says "Invalid 4D_FMRI" and gives the name of the fMRI it couldn't find. With that clue I found: https://git.fmrib.ox.ac.uk/fsl/fix/-/blob/master/hcp_fix#L38 You did not run the HCP Pipelines version of the hcp_fix script, you have run the one included with the FSL FIX download, which is not up to date. Use this launcher script and edit its variables, instead of composing your own hcp_fix command: https://github.com/Washington-University/HCPpipelines/blob/master/Examples/Scripts/IcaFixProcessingBatch.sh

Sorry, I usually run the IcaFixProcessingBatch.sh to finish noise reduction process and occur log of hcp_fix

Please do cd $HCPPIPEDIR; git status. The MODIFIED: YES line means you may have changed something you weren't supposed to.

The error message that our version of hcp_fix gives says "Invalid 4D_FMRI" and gives the name of the fMRI it couldn't find. With that clue I found:

https://git.fmrib.ox.ac.uk/fsl/fix/-/blob/master/hcp_fix#L38

You did not run the HCP Pipelines version of the hcp_fix script, you have run the one included with the FSL FIX download, which is not up to date. Use this launcher script and edit its variables, instead of composing your own hcp_fix command:

https://github.com/Washington-University/HCPpipelines/blob/master/Examples/Scripts/IcaFixProcessingBatch.sh

I seem to have understood what you mean, IcaFixProcessingBatch.sh calls the instruction of IcaFixProcessingBatch.sh. Before I downloaded FIX and extracted it to $HCPPIPEDIR ICAFIX file, overwriting the new version of hcp_fix. So this version is out of date, which is why this error occurs.

[RuiWang0805]

Thank you for your help and bother you again! @coalsont @glasserm When run HCP Pipelines version of the hcp_fix script a error occured: ERROR: hp = 0 not currently supported

[coalsont]

You should never be overwriting anything inside or copying anything into $HCPPIPEDIR. Any other tools it needs should be kept strictly outside it, regardless of whether they contain something with the same name as something in the pipelines. The only parts you should change are the variables in the Examples/Scripts folder (and even then, I would suggest making a copy of the ones you need outside $HCPPIPEDIR and only modifying your copies - this makes it easier to keep track of your settings if you have more than one dataset to run).

We recommend using MR FIX for our data (and whenever possible for other data, and we recommend acquiring fMRI data with runs that have opposed phase direction, to avoid having dropout overlap across all runs), and hcp_fix_multi_run is more feature-complete than hcp_fix. It should automatically be used by IcaFixProcessingBatch when you properly specify them in fMRINames and set ConcatNames for them.

[RuiWang0805]

You should never be overwriting anything inside or copying anything into $HCPPIPEDIR. Any other tools it needs should be kept strictly outside it, regardless of whether they contain something with the same name as something in the pipelines. The only parts you should change are the variables in the Examples/Scripts folder (and even then, I would suggest making a copy of the ones you need outside $HCPPIPEDIR and only modifying your copies - this makes it easier to keep track of your settings if you have more than one dataset to run).

We recommend using MR FIX for our data (and whenever possible for other data, and we recommend acquiring fMRI data with runs that have opposed phase direction, to avoid having dropout overlap across all runs), and hcp_fix_multi_run is more feature-complete than hcp_fix. It should automatically be used by IcaFixProcessingBatch when you properly specify them in fMRINames and set ConcatNames for them.

Thank you for your help and bother you again! @glasserm @coalsont When run HCP Pipelines version of the hcp_fix script a error occured: ERROR: hp = 0 not currently supported ![Uploading error.png…]()