MATLAB R2014a compatibility

[jeanluct]

R2014 might have introduced somewhat different "mex" command interface. I am not sure if this is due to my local installation being faulty or not, but

#!shell
mex -v

does not output internal variables to the prompt anymore, making Makefiles as currently written ineffective.

If you have MATLAB R2014a installed (at UW-Madison it is available via DoIT for free), please confirm. I'll see if I can find a solution.

[jeanluct]

From Marko Budisic on 2014-05-05 15:13:27+00:00

After a bit of googling, this has been confirmed: Mathworks have changed the

#!shell

mex -v

interface. Creating an empty cpp file dummy.cpp and running

#!shell

mex -v dummy.cpp

now returns variables with a delimiter ":" versus "=" used by previous versions.

This affects these Makefiles in braidlab, as far as I can see.

#!shell

find . -name "Makefile*" | xargs grep '(MEX) -v' 
./+braidlab/+lcs/private/Makefile:MEXENV      = $(MEX) -v 2> /dev/null
./+braidlab/@braid/private/Makefile:MEXENV      = $(MEX) -v 2> /dev/null
./+braidlab/@cfbraid/private/Makefile:MEXENV      = $(MEX) -v 2> /dev/null
./+braidlab/private/Makefile:MEXENV      = $(MEX) -v 2> /dev/null
./extern/assignmentoptimal/Makefile:MEXENV      = $(MEX) -v 2> /dev/null

[jeanluct]

From Marko Budisic on 2014-05-05 16:03:55+00:00

It looks like R2014a has a shell script

#!shell

mexsh

that mimicks the behavior of the mex in previous versions. Therefore I believe Makefiles should be changed to use mexsh if it exists, otherwise use mex for $(MEX) variable. I believe command -v foo provides a test in bash whether a command exists or not.

[jeanluct]

From Marko Budisic on 2014-05-05 16:43:02+00:00

R2014a compatibility in makefiles. Fixes Issue #57

→ <<cset 11722957112d>>

[jeanluct]

From Jean-Luc Thiffeault on 2014-05-21 19:58:59+00:00

I just installed R2014a and encountered this issue. I had to create a link to mexsh since it doesn't do so by default. We need to document this better or update to the new format.

[jeanluct]

From Marko Budisic on 2014-05-21 20:04:52+00:00

Variables in the new format are difficult to obtain, since the new mex does not support calling it without a file. I believe that the "official" Mathworks response on their forums was "create a dummy file and pass it to mex". If you open mexsh you'll see that the function does some funky business behind the screen. I'd suggest we add into the documentation that the user has to be able to execute mexsh on R2014a+ versions of Matlab for now.

[jeanluct]

From Jean-Luc Thiffeault on 2014-05-21 20:07:08+00:00

But is there a way to warn the user, without them having to read the doc?

[jeanluct]

From Marko Budisic on 2014-05-21 20:14:08+00:00

I think so, although it might involve a bit of bash/Makefile magic. The commit that initially closed this issue involved identifying the mex command using

MEX = $(shell { command -v mexsh || command -v mex ; } 2>/dev/null )

in Makefiles. We could modify this to read

MEX         = $(shell { command -v mexsh || command -v mex || {echo "Cannot detect mex or mexsh commands for  the MATLAB installation. Please see braidlab documentation."; exit 1} } 2>/dev/null ) 

or something like that (I don't know exactly how this can be done off the top of my head).

[jeanluct]

From Jean-Luc Thiffeault on 2014-05-21 20:18:08+00:00

That could work. It's actually remarkably difficult to get the Matlab version number from the command line. Maybe

#!bash

matlab -nojvm -nosplash -r "version, quit"

but that actually requires starting the Matlab engine! Terrible.

[jeanluct]

From Marko Budisic on 2014-05-21 20:19:12+00:00

OK, I just tried changing the mexsh and mex to foo and bar just to see what happens when the Makefile fails. I get the error in Makefile

Makefile:56: *** Matlab "mex" command is not in your path.  Stop.

Which is emitted by Makefile $(error ) command. Perhaps we can just modify this line?

[jeanluct]

From Marko Budisic on 2014-05-21 20:26:09+00:00

Yes, the closest non-engine thing I could find is

matlab -n | grep -om1 "R2...."

which matches the first R2xxxx string in various options that Matlab counts on. I don't know how portable this is to Windows. It works on Mac OSX R2014a and linux R2013b

[jeanluct]

From Jean-Luc Thiffeault on 2014-05-21 20:30:31+00:00

Yep, portability is a problem. It would be way better if it returned the numerical version, as the "version" command does from within Matlab.

[jeanluct]

From Marko Budisic on 2014-05-21 20:31:54+00:00

Agreed. It seems as if they are trying to make it more difficult to write platform-portable Matlab... terrible.

[jeanluct]

From Marko Budisic on 2014-05-21 20:40:40+00:00

I've just discovered an additional wrinkle in the mex vs mexsh in R2014.

mex command actually runs mexsh if ./.matlab/R2014/mexopts.sh file is present, i.e., if Matlab detects that the old shell-style configuration is used instead new xml-style configuration. I found this on accident, trying to get mex to fail with limited output, like the first time I ran R2014, and I couldn't. In fact, it behaved exactly as we got used to in previous versions. But, if you remove mexopts.sh, it's back to the newfangled interface.

[jeanluct]

From Marko Budisic on 2014-05-21 20:44:39+00:00

Agh, if I remove mexopts.sh and therefore use "fresh" R2014a installation, "mexsh" version of Makefile does not work out of the box. For some reason, default Mac OS X target version is set as 10.7 somewhere in Matlab configuration which makes the compiler throw up.

[jeanluct]

From Marko Budisic on 2014-05-21 20:49:02+00:00

Here's the relevant discussion on Matlab forums

[jeanluct]

From Jean-Luc Thiffeault on 2014-05-21 21:24:57+00:00

Pretty awful...

[jeanluct]

From Jean-Luc Thiffeault on 2014-05-21 21:26:05+00:00

Should we start a thread on Mathworks about the best way to write multiplatform Makefiles? Someone might have a better idea, or some of the Mathworks people might chime in.

[jeanluct]

From Marko Budisic on 2014-05-21 21:28:20+00:00

Yes, I think that's a good idea. Do you want to write the initial post or should I do it?

[jeanluct]

From Jean-Luc Thiffeault on 2014-05-21 21:34:26+00:00

I'll let you do it if you don't mind: you have a better grasp of how messed up things have gotten.

[jeanluct]

From Marko Budisic on 2014-05-21 21:35:36+00:00

OK, I'll post link here once I write it up (I won't get to it right away, though).

[jeanluct]

From Marko Budisic on 2014-05-22 15:13:52+00:00

Here's the link to my Mathworks forum question.

[jeanluct]

From Jean-Luc Thiffeault on 2014-05-22 16:02:25+00:00

Well-explained!

[jeanluct]

From Marko Budisic on 2014-05-29 18:51:25+00:00

I've put in a Mathworks Service Request for this matter. Their website says that the processing time is 1 business day, so I'll let you know what they say.

[jeanluct]

From Jean-Luc Thiffeault on 2014-09-21 22:35:14+00:00

This is another thing we need to resolve for 2.0...

It seems to me the best way might be to not get anything from mex -v. Maybe use the mex command directly to compile the C++ code as they suggested. That still leaves the matter of determining the target extension (.mexa64, etc) and the matlab directory, but maybe we don't need that. In particular if we use the mex command directly maybe we don't need to know the mex directory?

I'll experiment a bit.

[jeanluct]

From Jean-Luc Thiffeault on 2014-09-21 23:07:26+00:00

I tried and it works... sort of. It does simplify the Makefile. However, to build libtrains and libcbraid we need to use the same compiler that mex uses, so we still need to get that somehow.

(One advantage of the 'dummy' target is that it allows us to get the extension.)

[jeanluct]

From Jean-Luc Thiffeault on 2014-09-22 15:23:55+00:00

There are not too many extensions to check for: ext = mexext('all'); ext.ext gives mexglx, mexa64, mexmaci64, mexw32, mexw64. Funny that there's no 32-bit Mac. Probably just deprecated.

[jeanluct]

From Jean-Luc Thiffeault on 2014-09-22 22:19:10+00:00

Ok, I tried to completely rewrite the Makefiles in f7b30c5. Can you check that it works for the Mac? I'm not sure I got the right syntax in the root-level Makefile. I use uname to determine the system.

One drawback: currently this means that you can't run make in a subdirectory, only from the root directory which builds everything.

[jeanluct]

From Jean-Luc Thiffeault on 2014-09-22 22:29:01+00:00

By the way: I changed the check for BRAIDLAB_USE_GMP to explicitly check for =1, rather than just defined. The reason is that the manual states that you set it =1, so a user might justifiably believe that =0 should turn it off.

[jeanluct]

From Marko Budisic on 2014-09-23 20:36:45+00:00

The latest commit works on Mac Matlab R2014a and on R2014b with the old-style mex function.

On R2014b with new-style mex function compiler fails. At this point I cannot remember if I played with any other configurations, but I believe this is an out-of-box R2014b that is currently on my laptop.

cd +braidlab/private; make MEX=mex MEXSUFFIX=mexmaci64 MEXFLAGS="-largeArrayDims -O -DBRAIDLAB_USE_GMP" CXX="c++" CC="cc" CFLAGS="-O -DMATLAB_MEX_FILE" CXXFLAGS="-O -DMATLAB_MEX_FILE -std=c++0x -fPIC" GMP_LD="-lgmpxx -lgmp" all
mex -largeArrayDims -O -DBRAIDLAB_USE_GMP CFLAGS="-O -DMATLAB_MEX_FILE" CXXFLAGS="-O -DMATLAB_MEX_FILE -std=c++0x -fPIC" randomwalk_helper.cpp
Building with 'Xcode Clang++'.
Undefined symbols for architecture x86_64:
  "___sincos_stret", referenced from:
      _mexFunction in randomwalk_helper.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[jeanluct]

From Jean-Luc Thiffeault on 2014-09-23 20:49:40+00:00

Where did you get 2014b? On the Doit store I only see 2014a.

Does manual mex compilation work? randomwalk_helper should be pretty simple.

[jeanluct]

From Marko Budisic on 2014-09-23 21:38:06+00:00

I have a trial license from Matlab. They sent me a offer to try it out sometimes in spring, and the license expires in 5 days or so.

Running mex randomwalk_helper.cpp from command line produces a mexmaci64 file as expected.

[jeanluct]

From Jean-Luc Thiffeault on 2014-09-24 22:47:26+00:00

I think we're all good. Marko I think still has some issues but we'll reopen if needed.