MotionComp issues

[ryanelson1996]

Hey Denis,

I tried running motion comp on my laptop (Windows), and rather than it producing nifti files it has produced .hdr and .img files, instead.

Also it is causing problems with my command window. The command window itself has gone blank and the text cursor is two thirds of the way down the page but I can't type anything and it doesn't seem to be doing anything else. Tried the motion comp twice and it has done the same thing both times.

Any ideas what might be the issue?

Many thanks, Ryan

[schluppeck]

Hi ... two things here

1. nifti file format (whether you store as hdr/img pair or .nii files is controlled by a preference

mrGetPref() % to see status
% and to set to nifti
mrSetPref('niftiFileExtension', '.nii')

2. the command window being hogged is likely due to some mex file issues, especially if you are running on Windows, where the mex compiled files are probably not included for your platform. @julienbesle has done this, I think, and @elliohow has had some success running mrTools . mrAlign (?) under Windows.

Also see some hints on mex compiling here: http://gru.stanford.edu/doku.php/mrtools/gettingstarted

hth

[julienbesle]

Hi Ryan,

I think the command window issue is coming from mrDisp.m. A recent commit deleted the mex version of this file, but the matlab version causes some Java problems on Windows and Linux. I have fixed this in my fork. You can either clone my fork and use it instead of the original mrTools repository. Or you can just download the file mrDisp.mexw64 and place it in mrTools/mrUtilities/MatlabUtilities/ That should fix the problem. Let me know if not.

[julienbesle]

Actually, scratch the above. I didn't realize you're using the nottingham-neuroimaging branch, in which mrDisp.mex64 should still exist. The issue might still be caused by mrDisp.m though. What do you get if you type which mrDisp in Matlab?

@schluppeck the nottingham-neuroimaging branch is quite behind both Justin's and my branch. Should I merge all the recent commits into it?

[ryanelson1996]

Hi Julien and Denis,

Thank you both for your help.

MotionComp is now spitting out nifti files (thanks @schluppeck ).

I haven't looked into re-compiling the mex files yet but thought I'd run the motion compensation and get some screenshots. I have run it a few times with different effects, on the last attempt since doing a git pull of the mrTools repo it got stuck on the following display and wouldn't let me type on the command line at all:

Prior to the git pull it also did things like this: (but I've only run it once since so these things might happen again)

@julienbesle , here is an image of the output when I typed help mrDisp. which mrDisp had the path labelled with 'mrTools' so not particularly helpful but I did a git pull just before which I think confirms it was pulled from your fork?(https://github.com/julienbesle/mrTools)

(I hope these images load properly)

Many thanks, Ryan

[julienbesle]

hi Ryan,

Yes, your screenshots did load properly. The problems you get in the command window really do look like an issue with mrDisp. I need the output of the following command using which, because I need to know whether Matlab is using mrDisp.m or mrDisp.mexw64: which mrDisp -all

Also, to clarify, did you originally clone mrTools from https://github.com/justingardner/mrTools or from some other repository (and if yes, which one?) And did you obtain the above screenshots after switching to https://github.com/julienbesle/mrTools, or after doing a git pull from the mrTools repository you originally pulled from?

Cheers Julien

[ryanelson1996]

Hi Julien,

Here is the output for which

I originally cloned from https://github.com/justingardner/mrTools a while back but then also separately cloned https://github.com/julienbesle/mrTools which is what I was using when I took the screenshots.

Many thanks, Ryan

[julienbesle]

Ok. So it looks like Matlab is using the mex version of mrDisp. That's odd because I had a very similar problem with the red Java errors, but switching from mrDisp.m to mrDisp.mexw64 fixed it. I think it might be worth trying to recompile the mex version on your computer and see if that fixes the problem. First you could try and see what happens if you rename mrDisp.mexw64 temporarily so that Matlab uses mrDisp.m instead. Do you get the same errors or something different?

[julienbesle]

By the way, what version of matlab are you using?

[ryanelson1996]

There were numerous mex files, including a mexw32 and a matlab mex, but not an obvious .mexw64, so a put an underscore in all of the and tried again. which -all now just comes up with the mrDisp.m file. I didn't get the red java errors but did get the boxes and numbers again and now can't use the command line.

I'll have a crack at recompiling the mex version later and see if that helps.

Using Matlab 2019a, which I think is version 9.6

[julienbesle]

That's strange. If you got back mrDisp.mexw64 with the which command, then there should be an mrDisp.mexw64 file in that folder. Are you certain you looked in the same mrTools folder that is on the Matlab path (C:\Users\rjarc\Github_Repos\mrTools) ?

If yes, then the only other thing I can think of is that you're using a 32 bit version of Windows? Because Matlab only exists in the 64bit version, maybe it displays mrDisp.mex64 in the which output, but uses mrDisp.mex32 because it's on a 32 bit system?

In any case, I would say re-compiling mrDisp.c on your system might very well fix the problem.

Julien

[ryanelson1996]

It looks like the file explorer just calls it mrDisp with matlab mex under the file type, because within matlab it comes up saying there is a mrDisp.mexw64 file.

Checked in the settings and definitely running 64 bit version of Windows.

I've attempted to recompile the mrDisp.c file, which it said was successful, but still no luck on getting motion comp to run without any problems. Will have another crack at recompiling tomorrow a different way and see if that works.

Many thanks, Ryan

[schluppeck]

hi @julienbesle @ryanelson1996 ... was just thinking: often issues arise because of version jumps in matlab (have had some stuff break pretty much every time I upgraded matlab, esp where mex compiling is involved).

one thing to try first (as intermediate fix?!) would for ryan to use the same version of Matlab that @julienbesle @elliohow and others using under Windows 10... that would eliminate some of the hard to track down issues

so to end on a question: which version of matlab / Windows 10 are people using ?

version
ver

[julienbesle]

I'm using Matlab 9.1, 9.2, 9.4, and 9.5 on different Windows machines (and version 9.7 on Linux). Windows version is mostly 1903.

Do you get the problem with the matlab command window display only when running the motion comp analysis or when doing other things in mrLoadRet?

Which of the problems described above do you get when running with the compiled mrDisp.mexw64? Red java errors or not being able to type in the command window (or other)? And which do you get when running with mrDisp.m?

[ryanelson1996]

Good shout Denis, definitely worth a shot, I'll downgrade to Matlab 9.5, start from scratch and see if that runs okay.

I'm quite new to mrTools so haven't really done too much yet but I did also get the red java errors and some orange warnings when averaging the timeseries data and doing the correlation analyses in the retinotopy tutorial, but the analyses still worked and I don't think it hogged my command line at those times.

When running motion comp with the 'recompiled' version on mrDisp.mexw64 I still got the red java errors but I could still use my command line, when running with mrDisp.m I didn't get the red errors but did get the long lines of boxes and numbers and couldn't use the command line.

[julienbesle]

Until recently I was having the "red java issue" using mrDisp.m and it completely stopped when I started using the compiled mex file... which is the opposite of what you get. This was with Matlab version 9.4 But my students have been using versions 9.1 and 9.5 without any issue with either version of mrDisp (I think). And just tested with Matlab 9.1 with mrDisp.mexw64 and got no issue. So the issue might be Matlab version-dependent, but it could just as well be Java-version-dependent...

[ryanelson1996]

I've downloaded matlab 9.5 and am re-doing the retinotopy tutorial, I still get the red java errors and orange warnings when average timeseries with mrDisp.mexw64, but only get the orange warnings with mrDisp.m, so potentially looking promising, will update you when I try to run motion compensation as that's been causing me the biggest problems.

Edit: Spoke too soon, got the red java errors when doing the correlation analysis. The analysis still seemed to run properly though.

Edit number 2: Also came up with all the red java errors during motion compensation, even when using the .m rather than the .mexw64 version

[schluppeck]

@schluppeck the nottingham-neuroimaging branch is quite behind both Justin's and my branch. Should I merge all the recent commits into it?

I think this would be a good idea - worth checking out in the diffs whether there is anything in the nottingham-neuroimaging branch that needs to be kept. In general, probably a good idea to steer people either to your @julienbesle or main branch to avoid problems with too many different versions.

[schluppeck]

Edit: Spoke too soon, got the red java errors when doing the correlation analysis. The analysis still seemed to run properly though.

Edit number 2: Also came up with all the red java errors during motion compensation, even when using the .m rather than the .mexw64 version

this whole running on windows thing with multiple java / matlab / windows build versions is a bit messed up.

my suggestion for unpicking this is to make a reprex (as per https://www.tidyverse.org/help/)... basically the simplest version of a snippet of code that produces the problem (so not tied to running a tutorial/analysis/with data, just simple function calls in matlab with or without the mex files on the path), maybe:

1. with mrDisp.mexw64 on path:

mrDisp('test')
mrDisp('test\n')
mrDisp('test\b')
% etc

2. the same with mrDisp.mexw64 not on path, so essentially running fprintf

This might well be just a text, special character, format printing issue, but I am generally also a bit worried about other mex compiled functions (there is a fast interpolation function myCinterp3 and others that are needed for things to work correctly.

Given that we might be stuck in COVID-ville and everyone getting on macos or linux is not an immediate options, worth figuring out what's going on (even if a bit annoying).

[julienbesle]

@schluppeck the nottingham-neuroimaging branch is quite behind both Justin's and my branch. Should I merge all the recent commits into it?

I think this would be a good idea - worth checking out in the diffs whether there is anything in the nottingham-neuroimaging branch that needs to be kept. In general, probably a good idea to steer people either to your @julienbesle or main branch to avoid problems with too many different versions.

Ok. Will do. Not sure anymore why there was nottingham-specific repository. Maybe the idea was that it's a more stable version of my repository and I should only update it with commits I have been using for a while. I'll do this.

[julienbesle]

Just a few additional thoughts about the mrDisp/mex issue with Windows:

• I've been assuming from the start that the issue with with mrDisp, but it might well be with something else (although it seems mrDisp is at least partly involved).
• I've been using mrTools for many years on different Windows machines using different versions of Matlab and didn't have any Java issue until I merged Justin Gardner's changes in which he had removed the mex version of mrDisp. But I only got these errors on a single machine, and my students didn't get the same thing on other Windows machines. Also, it's not clear the Java errors I got were the same as what Ryan is getting now. On the other hand I also did get similar Java errors on Ubuntu, which went away when I put back the mex version of mrDisp... so it's not clear it is a Windows issue.
• at this point, I think it would be worth using Matlab's debugger to narrow down exactly when the issue occur (I guess that's a necessary step to make a reprex, but without assuming that the issue is with mrDisp). @ryanelson1996 i suggest you step through the mrTools code using "step" and "step in" buttons in the debugger until you get the Matlab command issues you've been having. Let me know if ou need any pointers on how to do this.

Can you post a screen capture of the "orange warnings" you get when using mrDisp.m?

[ryanelson1996]

Hi Denis and Julien,

Thanks for your suggestions, I will give these a crack, might take me a little while as I am fairly new to Matlab and programming in general but I'll give them both a shot.

These were the orange messages I was getting from when averaging the timeseries in the retinotopy tutorial, they came up when using both mrDisp.m and mrDisp.mexw64

Many thanks, Ryan

[julienbesle]

These warnings are unlikely to be related to mrDisp. I think this might be happening because something went wrong earlier in the tutorial and averageTSeriesGUI.m is missing some information to display its dialog window properly. Can you send a screen capture of the averageTSeries dialog window? In any case, the solution might simply be to start the tutorial from scratch.

[ryanelson1996]

I have started the retinotopy tutorial from scratch and am using Matlab 9.5 and will detail any problems as I go as I think a full account might be useful in case you pick up on anything. I started the tutorial from scratch twice up to the point of doing the correlation analysis, once with mrDisp.m and once with mrDisp.mexw64 on the path. I got the same errors and warnings both times.

First thing I noticed was a message (which I don't recall seeing with Matlab 9.6) that said I do not have a few toolboxes required (Image Processing Toolbox and Optimization Toolbox) so I have now downloaded those, but when I did quit the mrLoadRet session to do this I got the following warning message:

When I open the average timeseries window from the mrLoadRet gui I get the following:

Is this what you mean by the averageTSeries dialog window?

When I ran the average timeseries on the first set of runs (the wedges) it then came up with a the red java errors again even though last time I ran it with the .m and not the .mexw64 file I didn't get the java errors. Strangely when I ran it on the second set of runs (the rings) I did not get any errors.

When I opened up the correlation analysis gui I also got the warning messages:

(This image was taken before I realised I hadn't changed groups to the averages, but I got the same warning messages then too).

Running the correlation analysis again caused lots of java errors but did seem to output the results as expected.

I'll attempt to see if I can isolate which particular line in the analyseTSeries and correlation analysis scripts are causing the java errors

[julienbesle]

but when I did quit the mrLoadRet session to do this I got the following warning message:

I also saw these warnings before, but ignored them. I think you can do the same. I'll look into them though next time I see them

[julienbesle]

Is this what you mean by the averageTSeries dialog window?

Yes. It seems all the controls are displayed correctly in this dialog window, so I'm not sure why you're getting these warning messages. I think you can also ignore those too.

[julienbesle]

I'll attempt to see if I can isolate which particular line in the analyseTSeries and correlation analysis scripts are causing the java errors

First, put a breakpoint at line 65 of averageTSeries.m:

And then start the analysis from the GUI. Execution will pause at line 65:

You should then keep pressing the "Step" button while monitoring the command window for Java errors.

It's possible (likely) that the errors will occur while executing a subfunction within averageTSeries.m, in which case you'll have to use the "Step in" button to go inside that function. In that case, you'll have to start over and press step in when the green arrow in on the line of that subfunction. You can then keep pressing "Step".

Let me know at what stage of the averageTSeries.m (or subfunction) the Java errors start. You can do the same for function corAnal.m

[schluppeck]

May be related to matlab path issues under windows?!

Many error messages complain about escaped character \U

Eg https://uk.mathworks.com/matlabcentral/answers/435377-warning-escaped-character-u-is-not-valid

sprintf , fprintf etc would all show similar with windows \

This also reminds me of an older issue with saving mrDefaults.mat which goes at home / ~ on nix Systems.

Just thinking aloud — maybe shadow disppercent although there might be some other issues casting graphics objects to figure numbers (old style handles)

[ryanelson1996]

Okay haha, I will ignore the warning messages.

Thanks for the tip on where to start with the breakpoint and the 'step'/'step in' debugging, I'm back in the office on a different computer today and tomorrow so will have a crack at this next week probably when I'm back to working on my laptop.

How would I shadow disppercent? I tried googling this but didn't seem to find any thing helpful other than making a function with the same name?

Many thanks, Ryan

[julienbesle]

Shadowing means just that: having two or more functions with the same name on Matlab's path. Matlab will always use the one that comes first.

For instance, if I type:

>> which mrDisp -all
C:\Users\julien\Documents\matlab\mrToolsGit\mrUtilities\MatlabUtilities\mrDisp.mexw64
C:\Users\julien\Documents\matlab\mrToolsGit\mrUtilities\MatlabUtilities\mrDisp.m       % Shadowed

I can see that mrDisp.mexw64 "shadows" mrDisp.m, meaning that it will always be used when calling mrDisp. In this case they have different extensions, but the same would happen if they were in different folders and both called mrDisp.m.

In any case, the warning is issued on line 40 of mrSaveView.m, when trying to display the name of the "home" folder, where the data are saved. So if you could display this name, maybe we'd know what character in it might be causing the warning

You can get the name with the following commands (to use while mrLoadRet is open):

>> thisView = getMLRView;
>> viewGet(thisView,'homeDir')

ans =

C:\Users\julien\Documents\data\PTD\PTD01

>> clear thisView

But I wouldn't worry too much about this warning, which probably doesn't affect anything that mrTools is doing.

[ryanelson1996]

So haha...I tried the breakpoint and ran through averageTSeries step by step a couple of time....didn't get any red java errors, but then did get them when I ran the same analysis without doing the breakpoint, is it possible that the speed of the process is somehow causing the errors? I'll have a run through the correlation analysis and motion comp as well and see if I get the same there.

Here's the output from viewGet

`

thisView = getMLRView; viewGet(thisView, 'homeDir')

ans =

'C:\Users\rjarc\Desktop\Github_Repos\mrTools_tutorials\Retinotopy\retinotopyTutorial'

` Many thanks, Ryan

[julienbesle]

I tried the breakpoint and ran through averageTSeries step by step a couple of time....didn't get any red java errors, but then did get them when I ran the same analysis without doing the breakpoint, is it possible that the speed of the process is somehow causing the errors?

I thought this might happen. These Java errors are difficult to debug because they're not easy to replicate. Interestingly this page suggests that introducing pauses in the code could help get rid of them http://undocumentedmatlab.com/articles/handling-red-java-console-errors But I'm not sure where in the code these lines would have to be added. I'm not getting these errors at all any more, so I wouldn't be able to test.

Usually these java errors do not stop mrTools from running and doing what it's supposed to do. So if they do not prevent you from using the command window, I would just ignore them...

[julienbesle]

'C:\Users\rjarc\Desktop\Github_Repos\mrTools_tutorials\Retinotopy\retinotopyTutorial'

The warning occurs because fprintf in mrDisp.m interpret '\U' in '\Users' as an unknown special ('excaped') character. This only occurs in Windows when using mrDisp.m instead of mrDisp.mexw64. I'm not sure it's worth trying to fix as it doesn't really affect anything other than printing in the command window

[ryanelson1996]

Okay, pauses would make sense if it is something to do with the speed of the process, but I'll just ignore the messages if they don't hog the command window. I'll try the motion compensation as that caused me problems with using the command line and came up with the lines of boxes as well which I didn't get in the averaging or correlation analyses.

Edit: Ran motion compensation again using mrDisp.mexw64 and using the breakpoint and steps, same happened again, java errors came up when I ran though some of the loops quickly using 'run to cursor' with cursor set just outside the loop, but if I strolled through step by step it ran fine without any errors. Command line still worked as well. Previously using mrDisp.m rather than mexw64 it has come up with boxes and has stopped me from using the command line, so perhaps I'll stick with the mexw64 version and ignore the red warnings.

[schluppeck]

hi everyone - something to try on Windows platform to narrow down where java issues occur (my hunch is display / command window problems, etc)

for me:

>> which disppercent -all
/Users/lpzds1/matlab/mgl/utils/disppercent.m
/Users/lpzds1/matlab/mrTools/mrUtilities/MatlabUtilities/disppercent.m  % Shadowed

this is ok on macos -- also ok on Windows?

global gVerbose
gVerbose = true
disppercent(-inf,'Doing stuff');for i =  1:30;pause(0.1);disppercent(i/30);end;elapsedTime = disppercent(inf);

if not, does the following fix it??

global gVerbose
gVerbose = false
disppercent(-inf,'Doing stuff');for i =  1:30;pause(0.1);disppercent(i/30);end;elapsedTime = disppercent(inf);

[julienbesle]

The first test works fine for me on Windows. However I have mrTools's disppercent shadowing mgl's disppercent:

>> which disppercent -all
F:\matlab\mrToolsGit\mrUtilities\MatlabUtilities\disppercent.m
F:\matlab\mgl\utils\disppercent.m                               % Shadowed

If I switch to MGL's disppercent, then I start having problems, but these are different from the ones described by @ryanelson1996. Basically, disppercent starts eating up everything that's already printed in the command window. Too many \b's?

In general, I think it's a bad idea to have functions with the same name in mrTools and MGL. I'm planning to rename mrTools's disppercent to mlrDispPercent.m. Will let you know when that's done

[julienbesle]

I'm planning to rename mrTools's disppercent to mlrDispPercent.m. Will let you know when that's done

I've pushed a new branch called "replaceDisppercent" in which I've made sure that all calls to disppercent within mrTools are replaced with calls to mlrDispPercent. To test it, fetch the branch (in my fork of mrTools) and switch from branch "current" to branch "replaceDisppercent"

git fetch
git checkout replaceDisppercent

@ryanelson1996 , I'm not sure this is going to fix your issue, but it might... let me know

[ryanelson1996]

which disppercent -all showed that the mgl version was shadowing the mrTools version for me

doing the first test seemed to display something but then also delete a load of lines (too be back to which disppercent -a was this what it was meant to do?

I have done the git fetch and swapped to the "replaceDisppercent" branch, and ran the averaging timeseries and correlation analyses in the retinotopy tutorial, no red java errors in sight :D

[julienbesle]

Ok. Great! So the issue you were having was in fact with MGL's disppercent (@schluppeck good thinking!) and the mex-compiled version of mrDisp was probably working all along (but possibly not mrDisp.m). This would explain why I'm not getting the same error.

Could you keep using this branch and let me know how it's going as you keep using mrLoadRet?

If you don't get any more issues, I'll merge the new branch into my "current" branch.

[ryanelson1996]

Yeah of course, I've got a couple more tutorials to run through so I'll let you know whether or not I run into any problems :)

Thanks again @julienbesle and @schluppeck for all your help! :)

[schluppeck]

Looks like this may be solution for Windows/java related issues here, so closing this issue.

[julienbesle]

@ryanelson1996 Hi Ryan, Just wondering if you've encountered any issues with the branch I created in the last few weeks? I want to merge it into my branch. Thanks

[ryanelson1996]

Hey Julien, I haven't done much more fMRI analysis since but I haven't encountered any errors in the stuff I have done. Many thanks

[julienbesle]

Ok. I've merged the temporary "replaceDisppercent" branch into the "current" branch To go back to the current branch, you just need to do type:

git checkout current
git pull