Compilation on M1 Mac native

Edouard2laire commented 11 months ago

Hello,

Matlab 2023b is now running natively on computers with M1 chip and needs new MEX files (*.mexmaca64).

I compiled Mxcl by pulling from https://github.com/fangq/mcxcl and ran the benchmark and the performance is doubled in comparison to the previous version using Rosetta (matlab < 2023b) - although still quite slow: http://mcx.space/gpubench/. ( Score of 2270 vs 1002)

I uploaded the mex file here: https://drive.google.com/file/d/1WghdMRNOirwwx5-7fU3WPV4njpUsaQri/view?usp=share_link if you want to include it in the next release.

Regards, Edouard

>> info=mcxlabcl('gpuinfo')
Platform [0] Name Apple
============ GPU device ID 0 [1 of 1]: Apple M1  ============
 Device 1 of 1:     Apple M1
 Compute units   :  8 core(s)
 Global memory   :  11453251584 B
 Local memory    :  32768 B
 Constant memory :  1073741824 B
 Clock speed     :  1000 MHz
 Vendor name    :   Unknown
 Auto-thread    :   512
 Auto-block     :   64

Edouard2laire commented 11 months ago

Actually there is an error, I think with the generated Mex file as all the simulation will lead to NaN value:

[device 0(1): Apple M1] threadph=39062 extra=256 np=20000000 nthread=512 nblock=64 sharedbuf=1024 set kernel arguments complete : 3 ms lauching mcx_main_loop for time window [0.0ns 5.0ns] ... simulation run# 1 ... kernel complete: 138 ms retrieving flux ... transfer complete: 153 ms normalizing raw data ... source 1, normalization factor alpha=inf simulated 20000000 photons (20000000) with 1 devices (repeat x1) MCX simulation speed: 148148.15 photon/ms total simulated energy: 0.00 absorbed: nan% (loss due to initial specular reflection is excluded in the total)

Edit: I just tried to run the examples provided and only some of them work:

demo_mcxlab_2d.m : works
demo_mcxlab_basic.m: works
demo_mcxlabcl_basic: works
demo_validation_homogeneous: works
demo_continuous_mua_mus works
demo_4layer_head. works (log: https://justpaste.it/3wytq - but slow and second screen became unresponsive)
demo_colin27_atlas.m: doesn't work (log: https://justpaste.it/da22i)

Could it be some kind of memory issue? after running two times demo_colin27_atlas.m my entire computer crashed and automatically rebooted. Or maybe its an issue with openCL ?

Note: I am using OpenCL 5.3 and OpenGL 21.0.14

edit: i am getting very suspicious about this openCL version. From what i found online it should be version 3.0 edit2; i start to really hate opencl; never found a softaware with so little good information online.

Let me know if there is anything i should try. Edouard

Edouard2laire commented 11 months ago

So i added the following code in demo_colin27_atlas

cfg.debuglevel = 'T';
[T,detps]=mcxlabcl(cfg);
figure; mcxplotphotons(T)

here is the result:

simulation run# 1 ... 
kernel complete:    391 ms
retrieving flux ...     saved 519186 trajectory positions, total: 1519186   simulated 100000000 photons (100000000) with 1 devices (repeat x1)
MCX simulation speed: 259067.36 photon/ms
total simulated energy: 0.00    absorbed:   nan%
(loss due to initial specular reflection is excluded in the total)

Edouard2laire commented 7 months ago

Hello :)

Thanks again for providing such a great tool. Please find attached the output of the deploy code as you suggested : https://drive.google.com/drive/folders/1NMk2GaqsIVu7XuPOc8SwSMpJ96f_iBbg?usp=share_link

What would you like me to test to make sure everything is working?

if I try to run examples from Mcxl: I get a strange error:

[edelaire1@home-imglab01:~/Documents/software/mcxcl/mcxcl/example/quicktest]$ ./run_qtest.sh 
./run_qtest.sh: line 3:  4839 Killed: 9               ../../bin/mcxcl -A -n 1e7 -f qtest.inp "$@"

I think there is an issue with upx:

[edelaire1@home-imglab01:~/Documents/software/mcxcl/mcxcl/bin]$ ./mcxcl -L
Killed: 9
[edelaire1@home-imglab01:~/Documents/software/mcxcl/mcxcl/bin]$ upx -d mcxcl 
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2023
UPX 4.1.0       Markus Oberhumer, Laszlo Molnar & John Reiser    Aug 8th 2023

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
upx: mcxcl: CantUnpackException: file corrupted

Unpacked 0 files.

So I removed the upx line for the mcxl (upx -9 ../bin/mcxcl* || true) and seems to be running now:

[edelaire1@home-imglab01:~/Documents/software/mcxcl/mcxcl/bin]$ ./mcxcl -L
Platform [0] Name Apple
============ GPU device ID 0 [1 of 1]: Apple M1  ============
 Device 1 of 1:     Apple M1
 Compute units   :  8 core(s)
 Global memory   :  11453251584 B
 Local memory    :  32768 B
 Constant memory :  1073741824 B
 Clock speed     :  1000 MHz
 Vendor name    :   Unknown
 Auto-thread    :   512
 Auto-block     :   64

The issue with Colin27 is still present though :

[colin27.txt](https://github.com/fangq/mcxcl/files/14582052/colin27.txt)

fangq commented 7 months ago

thank you @Edouard2laire. I just realized that github had provided M1 processor access in the macos-14 runner since Jan 2024. so, I tweaked my github action script to build binaries. it works ok, with the exception that setup-matlab action fails to provide correct mex file link and failed the matlab mex build - see

https://github.com/matlab-actions/setup-matlab/issues/100

aside from that, it did successfully build both the binary and the octave mex file

https://mcx.space/nightly/github/mcxcl-macos-arm64-github-latest.zip https://mcx.space/nightly/github/mcxlabcl-macos-arm64-github-latest.zip

if matlab ci support team can fix the mex file issue above, we should be able to automatically build M1 binaries in the future.

will do the same for mmc.

fangq commented 7 months ago

mmc macos-14 build failed at this command

sudo ln -s /usr/local/lib/gcc/11/libgomp.a /usr/local/lib/libgomp.a

see log https://github.com/fangq/mmc/actions/runs/8259651723/job/22593909836

can you help me check what might happen for this command to fail? is there already a libgomp.a in that folder?

Edouard2laire commented 7 months ago

mmc macos-14 build failed at this command
sudo ln -s /usr/local/lib/gcc/11/libgomp.a /usr/local/lib/libgomp.a
see log https://github.com/fangq/mmc/actions/runs/8259651723/job/22593909836

can you help me check what might happen for this command to fail? is there already a libgomp.a in that folder?

I don't see any file called libgomp.a. The following code: find /usr -name "lib*gomp*lib" returns me :

/usr/local/gfortran/lib/libgomp.dylib
/usr/local/gfortran/lib/libgomp.1.dylib

fangq commented 7 months ago

libgomp.a should come with gcc installed via brew, which is provided by the macos-14 runner. although it seems the path has changed on m1 vs intel processors

fangq commented 7 months ago

@Edouard2laire, all packages for mcxcl, including mcxlabcl, have been successfully built for M1, please test it at https://mcx.space/nightly/github/

I still struggle with building mmc on M1, seeing various errors.

Edouard2laire commented 7 months ago

@Edouard2laire, all packages for mcxcl, including mcxlabcl, have been successfully built for M1, please test it at https://mcx.space/nightly/github/

I still struggle with building mmc on M1, seeing various errors.

i think upx is not working on arm mac:


[edelaire1@home-imglab01:~/Downloads/mcxcl]$ cd bin/
[edelaire1@home-imglab01:~/Downloads/mcxcl/bin]$ ./mcxcl -L
Killed: 9
[edelaire1@home-imglab01:~/Downloads/mcxcl/bin]$ ./mcxcl -L
Killed: 9
[edelaire1@home-imglab01:~/Downloads/mcxcl/bin]$ upx -d mcxcl 
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2023
UPX 4.1.0       Markus Oberhumer, Laszlo Molnar & John Reiser    Aug 8th 2023

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
   1064960 <-    770064   72.31%   macho/arm64   mcxcl

Unpacked 1 file.
[edelaire1@home-imglab01:~/Downloads/mcxcl/bin]$ ./mcxcl -L
Platform [0] Name Apple
============ GPU device ID 0 [1 of 1]: Apple M1  ============
 Device 1 of 1:     Apple M1
 Compute units   :  8 core(s)
 Global memory   :  11453251584 B
 Local memory    :  32768 B
 Constant memory :  1073741824 B
 Clock speed     :  1000 MHz
 Vendor name    :   Unknown
 Auto-thread    :   512
 Auto-block     :   64

The issue with Colin27 is also present in this version : https://justpaste.it/bobw9

fangq commented 7 months ago

I disabled upx on macos-14 and rebuilt the github ci binaries. For the issue you found with colin27, I will see if I can get an environment to debug this.

fangq commented 7 months ago

@Edouard2laire, I was also able to build mmc using the macos-14 runner. Please see all macos-arm64 packages in

https://mcx.space/nightly/github/

I will close this ticket for now as both mcxcl and mmc binaries are successfully built.

for the Colin27 error, I will open a new ticket.

fangq / mcxcl

Compilation on M1 Mac native #48