Compilation on linux

[bermanmaxim]

Hello,

I wanted to try this out, however I struggle compiling it on linux.

I have adapted the makefile as follows:

CC = g++
MEX_EXT = mexa64
JULIA_TOP_DIR  = /home/maxim/julia
JULIA_INC_DIR  = $(JULIA_TOP_DIR)/include/julia
JULIA_LIB_DIR  = $(JULIA_TOP_DIR)/lib/julia
TGT_DIR = ../m

MATLAB_INC_DIR = "/usr/local/MATLAB/R2014a/extern/include"
MATLAB_BIN_DIR = "/usr/local/MATLAB/R2014a/bin/glnxa64"

CCOPTS  = -I$(JULIA_INC_DIR) -I$(MATLAB_INC_DIR)
CCOPTS += -DMATLAB_MEX_FILE
REL_CCOPTS = -O3
DBG_CCOPTS = -g

COMPILE_FLAGS = -fPIC
LOPTS  = -L$(JULIA_LIB_DIR) -L$(MATLAB_BIN_DIR)
LOPTS += -shared -lmex -lmx
REL_LOPTS = -ljulia
DBG_LOPTS = -ljulia-debug

and added this -fPIC flag to the targets in the MakeFile, because g++ was issuing an error

/usr/bin/ld: jl_call.o: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC

without this flag. If I try to initialize the class Jl under Matlab, I get

Invalid MEX-file '/home/maxim/mexjulia/m/jl_calld.mexa64':
/home/maxim/mexjulia/m/jl_calld.mexa64: undefined symbol: jl_options

so somehow the jl_options struct symbol is not found, and if I comment these options out other jl_ symbols happen to be not found, etc...

What am I missing here ? Thanks !

[twadleigh]

I put together jlconfig.m, a matlab script for doing the configuration and build. It is currently in my fork of MATLAB.jl in the mex_interface branch, but parts of that will be migrating to this project soon.

It has the advantage of relying on MATLAB's mex command to utter the right incantation for your platform.

[bermanmaxim]

Thanks! I was able to make it work, but I had to modify your Makefile. It seems we don't have the same structure under julia: my JULIA_HOME is /home/maxim/julia/usr/bin, while julia_include_dir should be set to /home/maxim/julia/include/julia (no /usr), and my julia_lib_dir to /home/maxim/julia/lib/julia.

Moreover, I had to change the line cmd = '%s -e println(%s)'; to cmd = '%s -e ''println(%s)'''; to include quotation marks around my -e argument; and at runtime, at first it wouldn't find libjulia.so, so rather than adding julia library to LD_LIBRARY_PATH, I followed these instructions and set the rpath to point to the shared library location during mex compilation.

A few easy modifications... My versioninfo() is

Julia Version 0.4.0-pre+7105
Commit 349a4e1* (2015-08-30 15:16 UTC)
Platform Info:
  System: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

[bermanmaxim]

Well, while I was able to compile your mex_interface MATLAB.jl fork, I have segmentation faults closing MATLAB when I use eval for a second time; moreover it seems to me that julia is initialized at each execution of the mex file, and I would be rather looking for a solution that leaves julia running in the background...

[twadleigh]

The design is such that initialization should only occur once. Are you calling jlcall directly? Right now, that's dangerous. My intention was that users would only use the interface exposed by Jl.m, which would be able to handle all the pitfalls.

The initialize-once functionality is working for me on Windows. Indeed, Jl.call and Jl.eval wouldn't work at all if it didn't.

Of course, it took weeks of painful, low-level debugging to work out all the kinks to get this working consistently on my platform. You are the first to try this code base on Linux (or any other platform, for that matter). While I don't think there will be many weeks' worth, there may still need to be some low-level sleuthing that needs to be done to work out the kinks on your platform.

Thanks for giving this a go. By the end of this weekend I'm hoping to have this repo in such a state that I will be able to solicit the help of early adopters (like yourself, possibly) to test it out on other platforms. Then, hopefully, we can systematically work through remaining platform dependent issues.

[twadleigh]

The contents of the project now look the way I want them. If you get a moment, please give it an update, follow the README, and let me know if you are still having issues. I'll do what I can to help you get through them.

[bermanmaxim]

Hello, I will look into this in the next day, and try to help... Thanks for your work

[KristofferC]

I had to do the following for the .m script to run successfully on my ubuntu computer:

diff --git a/m/jlconfig.m b/m/jlconfig.m
index de29f9a..ef9344b 100644
--- a/m/jlconfig.m
+++ b/m/jlconfig.m
@@ -31,7 +31,7 @@ assert(exist(julia_bin_dir, 'dir') == 7);
 fprintf('The directory of the Julia executable is %s\n', julia_bin_dir);

 % get julia home
-cmd = '%s -e println(%s)';
+cmd = '%s -e ''println(%s)''';
 [~, julia_home] = system(sprintf(cmd, exe, 'JULIA_HOME'));
 julia_home = chomp(julia_home);
 assert(exist(julia_home, 'dir') == 7);
@@ -44,7 +44,7 @@ assert(exist(julia_image, 'file') == 2);
 fprintf('The Julia image is %s\n', julia_image);

 % get include dir
-[~, julia_include_dir] = system(sprintf(cmd, exe, '"joinpath(match(r\"(.*)(bin)\",JULIA_HOME).captures[1],\"include\",\"julia\")"'));
+[~, julia_include_dir] = system(sprintf(cmd, exe, 'joinpath(match(r"(.*)(bin)",JULIA_HOME).captures[1], "include", "julia")'));
 julia_include_dir = chomp(julia_include_dir);
 assert(exist(julia_include_dir, 'dir') == 7);
 assert(exist([julia_include_dir filesep 'julia.h'], 'file') == 2);
@@ -56,9 +56,10 @@ if ispc
   julia_lib_dir = strjoin(bits(1:end-2), filesep);
   lib_opt = 'libjulia.dll.a';
 else
-  [~, julia_lib_dir] = system(sprintf(cmd, exe, 'abspath(dirname(Libdl.dlpath(\"libjulia\")))'));
+  [~, julia_lib_dir] = system(sprintf(cmd, exe, 'abspath(dirname(Libdl.dlpath("libjulia")))'));
   lib_opt = '-ljulia';
 end
+julia_lib_dir = chomp(julia_lib_dir);
 assert(exist(julia_lib_dir, 'dir') == 7);

 % write the config file

[twadleigh]

@KristofferC, I applied your patch with some if/elses so as not to break the PC build. Would you mind checking and reporting back if the latest version of jlconfig is completing for you without need for patching?

[KristofferC]

Yes it works for me with your additions. I get this though, might just be my system:

Warning: Unable to save path to file
'/usr/local/MATLAB/R2015a/toolbox/local/pathdef.m'. You can save your
path to a different location by calling SAVEPATH with an input argument
that specifies the full path. For MATLAB to use that path in future
sessions, save the path to 'pathdef.m' in your MATLAB startup folder. 
> In savepath (line 169)
  In jlconfig (line 100) 

[twadleigh]

OK. I'll close this issue and open another related to improving the handling of saving the path when savepath fails.

[bermanmaxim]

Hello, The MEX script works if I select bin/julia under julia main folder. There is one catch though, my julia symbolic link under julia main folder points to a different julia executable, under usr/bin/julia: selecting this other executable breaks the make file, since it doesn't account for the extra usr/ in the path. Since this other executable is the one that is pointed to by which julia, "guessing the path" of julia does not work. Then when I try to use it, I have Invalid MEX-file '/home/maxim/jlcall/m/jlcall.mexa64': libjulia.so: cannot open shared object file: No such file or directory; as I said, this is related to my libjulia.so not being in my LD_LIBRARY_PATH... I added this folder to that path in matlab using setenv('LD_LIBRARY_PATH', [getenv('LD_LIBRARY_PATH') ':' '/home/maxim/julia/lib/julia']); prior to compiling and it worked. I think now it really works, I am able to eval multiple times, no segfault ;) Great work @twadleigh !

[bermanmaxim]

Correction: apparently it is the setenv('LD_RUN_PATH', '/home/maxim/julia/lib/julia'); command that really helped, not LD_LIBRARY_PATH... And I have to set it at each new instantiation of Matlab for it to work. I added this LD_RUN_PATH definition to the initialization of the Jl class, and it conveniently works.

[twadleigh]

I'm opening a separate issue for the LD_RUN_PATH thing.

[allenyin]

Then when I try to use it, I have Invalid MEX-file '/home/maxim/jlcall/m/jlcall.mexa64': libjulia.so: cannot open shared object file: No such file or directory; as I said, this is related to my libjulia.so not being in my LD_LIBRARY_PATH... I added this folder to that path in matlab using setenv('LD_LIBRARY_PATH', [getenv('LD_LIBRARY_PATH') ':' '/home/maxim/julia/lib/julia']); prior to compiling and it worked.

I got the same error, and this fixed worked -- i.e. setting LD_RUN_PATH to directory where libjulia.so is located. I'm using Ubuntu14.04 with Matlab2013b