Matlab/Paramonte fails to run on M1 Apple...

[arwelHughes]

• matlab R2023; M1 Apple mac; OS Sonoma 14.4.1*

I cloned the toolbox from Github, and tried to build the toolbox as:

./install.sh --lang MATLAB --build release --par omp

Then, I added the folder containing the '+new' folder to the Matlab path and saved.

Navigated to '/example/matlab/sampling/Paradram', and tried to run 'main.m', but received the following error:

main \brief This is the base class for the ParaMonte sampler routines.

\note See below for information on the attributes (properties).

\note See below for information on the methods.

\final{Sampler}

\author \JoshuaOsborne, May 21 2024, 12:43 AM, University of Texas at Arlington
\FatemehBagheri, May 20 2024, 1:25 PM, NASA Goddard Space Flight Center, Washington, D.C.
\AmirShahmoradi, May 16 2016, 9:03 AM, Oden Institute for Computational Engineering and Sciences (ICES), UT Austin

Documentation for pm.sampling.Sampler

libspecs = "darwin" "arm64" "shared" "heap" "serial"

bldtypes = "native" "tuned" "ipo" "release" "testing" "debug"

clstypes = "intel" "gnu"

Error using pm.sampling.Sampler/run

There are no MEX libraries associated with the configurations displayed above: Either the user has compromised internal structure of the ParaMonte library or the user has tempered with hidden attributes of the ParaMonte sampler. If you believe neither is the case, please report this error at:

https://github.com/cdslaborg/paramonte/issues

for a quick resolution.

Error in pm.sampling.Paradram/run (line 49) run@pm.sampling.Sampler(self, getLogFunc, ndim);

Error in main (line 12) sampler.run ( @(x) pm.stats.dist.himmelblau.getLogUDF(x(1), x(2)) ...

[shahmoradi]

@arwelHughes, Can you check if any *.dylib MEX files exist in the subfolders of the directory +pm/lib? If there is none, it means the CMake script did not generate any MEX file, probably because it did not find MATLAB. If so, type the following commands first on the command line:

export Matlab_ROOT_DIR="path to the matlab root directory, like: /Applications/MATLAB_R2024a"
export matlab="path to the MATLAB executable, like: /Applications/MATLAB_R2024a/bin/matlab"

Then retry the build command:

./install.sh --lang matlab --par "serial;omp"

Let me know if this works. Thanks.

[arwelHughes]

Okay, it does make mex/dylib, but interestingly it seems to have made intel based mex (mexmaci64) rather than Arm based mex (mexmaca64)...

However, I'm trying to run this on a native Apple Silicon version of R2023a (i.e. not the one that runs via Rosetta). I'll try to see if it runs on an older version of Matlab (which will go through Rosetta)....

[shahmoradi]

Can you confirm if you are not running the build script in Rosetta compatibility mode with the Intel chip? As a temporary solution, could you change the mexext matlab variable in this line to "mexmaci64" and retry the sampling example again? If this temporary solution works, it means the problem is just a wrong library extension choice by your MATLAB MEX compiler, which is still bizarre. But if it still does not work (because it cannot load the library because it is built for Intel chips), then it is likely related to Rosetta compatibility mode or something similar on the laptop. I have to note that all tests and examples run successfully on my M2 Mac.

[arwelHughes]

I'm trying to run it from the native Apple Silicon version of R2023b on an M1 mac, so there should be no Rosetta involved. In other words this release expects mexmaca64, and it won't recognise mexmaci64. So by the sounds of it the build is running in Rosetta compatability mode when it shouldn't be, and I need to set a flag somewhere?

[shahmoradi]

Could you try this in Rosetta compatibility mode and see if it works there? I have never used Rosetta compatibility mode, and I will not know what may be causing it until I test it again in M1. @theoneineed, do you know what may be causing this GitHub issue?

[arwelHughes]

"Could you try this in Rosetta compatibility mode...." - I'm not clear what you mean.. is this a build setting / flag?? I don't really know what you mean by Rosetta Compatabillity Mode or how to set it.

[theoneineed]

@shahmoradi I just did a fresh install on my system with cloned repository and it looks like I am getting the same error as @arwelHughes . When comparing the two folders(one generated now, and one that I had from couple weeks back, when it was working), this is the message I see.

My installation process was same as the last time. I also installed it native to Apple Silicon without using rosetta. @arwelHughes I think by "Rosetta compatibility mode", he means do the build process using terminal opened in rosetta (go to applications/utilities/terminal, right click, get info, there should be a checkbox there that says "open using rosetta") but I don't think that is the issue here.

I have attached a zipped folder here with the version of pm that works for me ( https://drive.google.com/file/d/1lkPsm7Gn4R-XuUlbnoK6VJ-TrNyAltAB/view?usp=share_link )

[shahmoradi]

@arwelHughes @theoneineed I just rebuilt the library from a clone of the latest version of th elibrry on GitHub on a macOS M2, and both serial and openmp versions seem to work fine, passing all tests. @arwelHughes Can you ensure matlab path is specified in your PATH enviromental variable? You can check this by inspecting the output of

echo $PATH

in your terminal. To be safe, you can try this command before rerunning the build script,

export PATH=/Applications/MATLAB_R2023b.app/bin:$PATH

assuming your MATLAB version is R2023b. Try building and testing the library for the serial mode first via,

./install.sh --lang matlab --par serial

Note that the example script for Himmelblau function searches for serial version of the library. To use the openmp version, you need to change the line, sampler.spec.parallelismNumThread = []; in the script to sampler.spec.parallelismNumThread = 0; or some other positive number. Keep in mind that openmp parallelism within MATLAB has an overhead of about a few milliseconds, meaning that your objective function evaluation must take more than a few milliseconds to see any benefits from openmp parallelism, because the entire parallelim is handled by MATLAB Parallelism Toolbox, which is more than 100 times slower (more costly communication overhead) than the MATLAB-independent MPI parallelism.

[arwelHughes]

Thanks Amir, I will try this when I get to work. But just to be clear, on your M2 mac, are you running the Native Apple Silicon version of Matlab (https://uk.mathworks.com/support/requirements/apple-silicon.html), or the Rosetta Intel emulated one (see the comment at the end of the above link).

The former expects mex files to be .maci64 and the latter expects .maca64, and they are not compatible - the build I tried made the Intel binaries...

[shahmoradi]

It's all for M2, both matlab and the build. I am traveling at the moment. I will test it also with an M1 once I am back later today. Given the complexity of the issue, I'd say build all your tests for the default case (serial as opposed to openmp) and run the tests. When I built the library for openmp, I got a similar error to yours for the first time (although the library names were correct) because the example expected to find the serial versions of the library because of the value set for parallelismNumThread. This could also be the reason for the error you see and the matlab extension name may have nothing to do with this. As a test, you can open matlab and type mexext to see what name matlab uses. Let us know your observations here. Thanks!

[fagheri]

@arwelHughes I have added a MATLAB arm64 pre-built package to the latest pre-release of the library on GitHub. While searching for the root cause of the error in this issue, please try the newly uploaded package and let us know if it also works for you. I have tested the package on both M1 and M2 ARM chips, and it seems to work fine. Here is the download instruction:

1. Open a terminal.
2. type,

   cd ~/Downloads
   curl -OL https://github.com/cdslaborg/paramonte/releases/download/2024.04.29/libparamonte_matlab_darwin_arm64.tar.gz
   tar -xvzf libparamonte_matlab_darwin_arm64.tar.gz && cd libparamonte_matlab_darwin_arm64/example
   matlab

3. Once the MATLAB session opens, try running the main.m file in the current working directory. This will run the sampler for the Himmelblau test function, among other examples it runs.

Let us know if this temporary solution works. Note that some of the visualization functionalities (e.g., Corner plots) are not fully implemented yet.

[arwelHughes]

Yes - that worked. Thanks!

[shahmoradi]

@arwelHughes This issue must have been fixed now in the latest pre-release of ParaMonte::MATLAB:

https://www.cdslab.org/paramonte/generic/latest/installation/matlab/#the-fast-way

Please download the pre-built libraries from this page and test with the sampling examples provided. If the error persists, reopen this issue or open a new one. We have now tested the latest pre-release, and it should be fully functional and much more convenient to work with than the past releases, particularly in relation to parallelism features, both MPI and now OpenMP (MATLAB Parallelism Toolbox).