Enable matlabneuroninterface on Linux

[AljenU]

Thanks to @thkupy for looking into this. To enable matlabneuroninterface on Linux:

• LD_LIBRARY_PATH need to be set outside Matlab already (see also https://nl.mathworks.com/help/matlab/matlab_external/set-run-time-library-path-for-c-interface.html)
• A definition of modl_reg needs to be added to our header

For the first item, I'd say we want to

• check whether LD_LIBRARY_PATH is correct in setup.m
• if not, generate a bash script that can do the modification
• andd have a clear error message to the user that says matlab needs to be started from a shell with the correct LD_LIBRARY_PATH, and point to the generated script as a helper

For the second

• add extern “C” modl_reg(){}; to our header
• it would be nice to be able to add an explanation in the header as to why this is needed.

[AljenU]

Additionally, as can be seen in the old linux tryout in #27 , the header needs some checking on name_mangling of functions. Below is an old example, the header will be using different functions in the most up-to-date main branch.

If also supporting Mac, then the #else will need to be expanded with an #elseif

#ifdef _WIN32
// Import C++ name mangled functions.
__declspec(dllimport) void ivocmain_session(int, const char**, 
                                            const char**, int);
// Import non-name mangled functions.
extern "C" __declspec(dllimport) int hoc_oc(const char*);
extern "C" __declspec(dllimport) void* hoc_lookup(const char*);
extern "C" __declspec(dllimport) double hoc_call_func(void*, int);
extern "C" __declspec(dllimport) void hoc_pushx(double);
#else 
// Import non-name mangled functions.
extern "C" void ivocmain_session(int, const char**, const char**, int);
extern "C" int hoc_oc(const char*);
extern "C" void* hoc_lookup(const char*);
extern "C" double hoc_call_func(void*, int);
extern "C" void hoc_pushx(double);
#endif

[AljenU]

And, the setup.m and the build_interface.m will need some ispc() checks, and different paths used, when doing linux instead of windows. Or when doing Mac instead of linux or windows.

[AljenU]

The basic changes have been made, and we are sort of able to run on linux without crashing immediately.

The necessary commits are in https://github.com/mcdougallab/matlabneuroninterface/tree/kian/neuron9dev-linux It requires Neuron 9 (edited)

However, there seems to be a fatal interaction with the Matlab jvm. After doing anything with neuron through clib, Matlab will then crash a bit later, usually when interacting with the Matlab GUI. The crash log then shows that "sys/java/jre/glnxa64/jre/lib/amd64/server/libjvm.so" within the matlab root directory is involved.

This even though only a single folder is added to the LD_LIBRARY_PATH, that contains very few libraries: libcoreneuron.so libcorenrnmech.so libcorenrnmpi_mpich.so libcorenrnmpi_mpt.so libcorenrnmpi_ompi.so libinterviews.a libivx11dynam.so libnrniv.so libnrnmpi_mpich.so libnrnmpi_mpt.so libnrnmpi_ompi.so libnrnpython3.so librxdmath.so libscopmath.a libunidraw.a

[AljenU]

We might need @tkuenzmw (edited from @thkupy) to have a look at this?

Where I said "After doing anything with neuron through clib", I meant running example_run, but also just clib.neuron.initialize();

Need help, because debugging matlab with jvm on linux is problematic. The matlab support page says to disable the jvm when debugging (https://nl.mathworks.com/help/matlab/matlab_external/debugging-on-linux-platforms.html). And indeed rightly so, because multiple SIGSEGV occur just when starting matlab itself with the jvm enabled.

[ramcdougal]

As part of the process, we'll want to make sure examples/example_mod.m works in all platforms. The original version assumes a Windows .dll file extension.

[AljenU]

@tkuenzmw I just saw that in 2023a, there is https://nl.mathworks.com/help/matlab/ref/clibconfiguration.html, which allows an out-of-process executionmode. Does the jvm related crash still happen when the lib config is set to out-of-process?

[tkuenzmw]

@AljenU I will explore that tomorrow. Today I am in the office and do not have a linux machine available. In 23a I can also test whether the crash occurs in the new javascript environment as well.

[tkuenzmw]

So this is confusing.

R2023a, regular (JVM) desktop: When I set the execution mode to "out of process" in R2023a, MATLAB refuses to load the interface library, although all paths are set up correctly. When I switch to "inprocess", the known crash behavior occurs

R2023a, -nodesktop: works in "inprocess" mode, but does not find library in "out of process mode"

R2023a, new javascript-based desktop: works in "inprocess" mode without crashes, but does not work in "out of process" mode again (not finding the library).

This confuses me. But that would mean, that in R2023a with the beta Desktop (javascript) it works.

Edit: update, I can still get it to crash with errors in the libjvm.so and the libc.so after using the interactive livescript a while (without errors) and then calling clib.neuron.get_nrn_functions().

[edovanveen]

I just did some tests on the main branch on Windows and if I move the sliders in the interactive example too quickly I also get a crash.

[ramcdougal]

Hypothesis: maybe if the sliders are moved too fast, it's asking NEURON to start a new simulation before the old one finished? Maybe there could be a flag for an in-progress simulation? You'd still want it to end at the right point, but you don't necessarily need to hit every intermediate value.

[AljenU]

@edovanveen @ramcdougal : I'd say the windows sliders is part of the random-crashes issue, not this Linux issue.

[AljenU]

@tkuenzmw I did get it running on Linux, 2023a, outofprocess. The missing dependency in that case is the libmex! With outofprocess, the environment does NOT inherit Matlabs additions to the LD_LIBRARY_PATH. And our interface library, in addition to libnrniv, also depends on libmex. Adding 'MATLABROOT/bin/glnxa64/' (with correct replacement for MATLABROOT) explicitly to the LD_LIBRARY_PATH before starting matlab, in addition to the path where the libnrniv is, makes that outofprocess does run.

At all: Using 2023a, and setting the clibConfiguration to outofprocess, the matlabneuroninterface works (kind of) correctly!

• Since it runs in a different process, the redirection of output printing to the command window does not work. The output is instead printed in the shell from where the matlab session was started.
• Getting the data to the Matlab session, and making plots based on that, does work. I.e. running example_acpot creates the correct figure.

While looking into this, I started from a minimal interaction with the neuron library. For inprocess, the crash starts to happen when ivocmain_session is called. If i uncomment that one from our cpp initialize() function, then I can run the initialize function from Matlab, and the Matlab session does not crash.

[tkuenzmw]

Hi Aljen, just out of curiosity: how did you find out? I used ldd -d within and outside of MATLAB… every dependency was resolvable.

Edit: for me this does not suffice. What are you adding to the LD_LIBRARY_PATH exactly? I add: the path to the neuron executables (just in case) the path to the libnrniv.so the path to the neuronInterface.so (just in case), matlabroot/bin/glxn64 matlabroot/sys/os/glxn64.

Still, in out of process execution mode it complains the module cannot be found. What could I be missing?

[AljenU]

I only had (with the PATH TO parts correctly set of course, and my conda environment is named neuron9):

export LD_LIBRARY_PATH='PATH/TO/matlabneuroninterface/neuron/:PATH/TO/.conda/envs/neuron9/lib/python3.11/site-packages/neuron/.data/lib/:PATH/TO/Matlab2023aInstall/bin/glnxa64/:'

[tkuenzmw]

@AljenU thank you, I figured it out, it works for me now as well. For some reason it would not accept the set LD_LIBRARY_PATH until I ran sudo ldconfig once. The whole business of setting the LD_LIBRARY_PATH could really be improved in MATLAB.

[AljenU]

@kian-ohara : For this issue, some edits to the branch are needed still.

In the README:

• Need to expand the warning at the top of the readme on what options are supported: Linux + gcc + Matlab2023a
• Need to update the Setup part with separate instructions for Linux

In setup.m:

• Do NOT rely on the user having used conda to install neuron, require a full path to be filled in. Put in a clearly-fake path in the commit.
• Check for libnrniv.so, not for bin/nrniv. And for the path to libnrniv.so being part of LD_LIBRARY_PATH.
• EDIT. ADDED: set out-of-process for neuron clib. Should be possible to already do even if not generated yet, I think.

In build_interface.m:

• Change variable name StaticLibPath to NrnLibPath
• Do NOT rely on the user having used conda to install neuron, require a full path to be filled in. Put in a clearly-fake path in the commit.
• The error("Error while running clibgen.generateLibraryDefinition(), please check if you have administrator rights.") should only be shown when ispc, else just rethrow(ME)

[ramcdougal]

@tkuenzmw How do I enable R2023a's "new javascript-based desktop"? I can't find any information about this.

EDIT: Found it. You have to go to add ins and search for "New Desktop" first. The new desktop looks slick, and it has the secondary benefit of preventing a (surprisingly harmless, except for a nasty message) Java NullPointerException from opengl on my machine.

[tkuenzmw]

@tkuenzmw How do I enable R2023a's "new javascript-based desktop"? I can't find any information about this.

EDIT: Found it. You have to go to add ins and search for "New Desktop" first. The new desktop looks slick, and it has the secondary benefit of preventing a (surprisingly harmless, except for a nasty message) Java NullPointerException from opengl on my machine.

Hi Robert, sorry I was slow in my reply. Good that you found it. There are many benefits of the new JS Desktop but also a few limitations: older, complex GUI and Apps might not be compatible (like in MATLAB Online).

[tkuenzmw]

Why do I get this error about indexing?

>> n.list_functions
Available variables:
    PI
    E
    GAMMA
    DEG
    PHI
    hoc_ac_
    float_epsilon
    hoc_cross_x_
    hoc_cross_y_
    default_dll_loaded_
    xopen_broadcast_
    clamp_resist
    celsius
    secondorder
    diam_changed
    nrn_shape_changed_
    nrn_netrec_state_adjust
    nrn_sparse_partrans
    stoprun
    t
    dt
    nai0_na_ion
    nao0_na_ion
    ki0_k_ion
    ko0_k_ion
    ib_IntFire4
    eps_IntFire4
    taueps_IntFire4
Available procedures (returning nothing):
Available variables:
    PI
    E
    GAMMA
    DEG
    PHI
    hoc_ac_
    float_epsilon
    hoc_cross_x_
    hoc_cross_y_
    default_dll_loaded_
    xopen_broadcast_
    clamp_resist
    celsius
    secondorder
    diam_changed
    nrn_shape_changed_
    nrn_netrec_state_adjust
    nrn_sparse_partrans
    stoprun
    t
    dt
    nai0_na_ion
    nao0_na_ion
    ki0_k_ion
    ko0_k_ion
    ib_IntFire4
    eps_IntFire4
    taueps_IntFire4
Available procedures (returning nothing):
Dot indexing is not supported for variables of this type.

Error in neuron.Neuron/list_functions (line 150)
            for i=1:self.fn_void_list.length()

Error in neuron.Neuron/dynamic_call (line 105)
                [varargout{1:nargout}] = builtin('subsref', self, S);

Error in indexing (line 139)
                [varargout{1:nargout}] = self.dynamic_call(S);```

[AljenU]

Because there is a small bug with initializing a property. Fix commit incoming.

[tkuenzmw]

That has worked. Could it be that a similar error prevents me from using quit() or am I mistaken? The error message

Error in neuron.Neuron/fill_dynamic_props (line 25)
            arr = split(clib.neuron.get_nrn_functions(), ";");

Error in indexing (line 137)
                self.fill_dynamic_props();

Error in testmpi (line 14)
n.quit();

says something about indexing as well...

[edovanveen]

On Windows, n.quit() simply causes MATLAB to close with no further errors shown.

[AljenU]

For me, quit() gives a sort of expected behaviour, I think? The entire neuron is cancelled, it looks like, see output below. Which does look like a similar error to the indexing one, because quiting neuron unloads the library, and then there are follow up messages because we did not expect that. We might have to make quit() a special case, to avoid confusing warnings, and automatically unload / reload the library?

It looks like neuron just shuts down the entire process on quit, which would explain why Matlab closes on Windows. There, neuron is running inprocess. On linux, neuron is running outofprocess, and we get MATLABCLibHost process for 'neuron' terminated unexpectedly. To reload interface library, first call "unload(clibConfiguration('neuron'))"

>> setup;
>> n=neuron.Neuron;
>> n.quit()
Warning: MATLABCLibHost process for 'neuron' terminated unexpectedly. To reload interface library, first call "unload(clibConfiguration('neuron'))"
and then call function/class from interface library. 
> In neuron.Neuron.call_func_hoc (line 211)
In neuron/Neuron/dynamic_call (line 110)
In indexing (line 133) 
Warning: 'quit': number or type of arguments incorrect. 
> In neuron.Neuron.call_func_hoc (line 212)
In neuron/Neuron/dynamic_call (line 110)
In indexing (line 133) 
Error using clib.neuron.get_nrn_functions
MATLABCLibHost process for 'neuron' terminated unexpectedly. To reload interface library, first call "unload(clibConfiguration('neuron'))" and then
call function/class from interface library.

Error in neuron.Neuron/fill_dynamic_props (line 25)
            arr = split(clib.neuron.get_nrn_functions(), ";");

Error in indexing (line 137)
                self.fill_dynamic_props();