Old compiled mex files (Fortran code) fails on Ubuntu 20.04+ & modern MacOS

thomas-nilsson-irfu commented 3 years ago

This issue is added here primarily to serve as a reference point if more people encounter it (and as a reminder to myself).

Step 1: Latest code?

[x] This issue is present in the most recent code (i.e. after running git pull this is still a problem).

Step 2: Describe your environment

irfu-matlab branch (i.e. "master", "devel"): devel
Matlab version used (i.e. "Matlab R2017a"): Matlab R2020, Update 1
Operating system (ie "Windows 7", "Mac OS X 10.12.3", "Linux Ubuntu 16.04"): Linux Ubuntu 20.04
[x] A 64-bit installation of Matlab was used.

Step 3: Describe the problem

The problem is that the old compiled mex files have been linked to an old libgfortran.so.3 which is no longer present in the Ubuntu's repositories for newer releases. Ubuntu 18.04 was the last one to include libgfortran3, newer releases have only libgfortran5 (which was also available for 18.04). The compiled mex files included in irfu-matlab must be recompiled on a newer system, however doing so will link them to libgfortran5 which is not available on older Ubuntu 16.04 systems thus breaking backwards compatibility.

I would prefer waiting until April 2021, when Ubuntu 16.04 has reached its "end of standard support" and essentially all Ubuntu users should have upgraded to at least 18.04 before updating the files here in irfu-matlab. If anyone disagrees, please add a comment!

Steps to reproduce:

irf; => IRBEM .. not OK. Please, contact irfu! or

tests_before_release =>

Error occurred in test_mms_spinfit/test_c_efw_spinfit_mx_simple and it did not run to completion.
---------
Error ID:
---------
'MATLAB:mex:ErrInvalidMEXFile'
--------------
Error Details:
--------------
Invalid MEX-file '/home/thoni/MMS/irfu-matlab-master/irfu-matlab/mission/cluster/c_efw_spinfit_mx.mexa64': libgfortran.so.3:
cannot open shared object file: No such file or directory

Error in test_mms_spinfit/test_c_efw_spinfit_mx_simple (line 42)
        c_efw_spinfit_mx(3,10,3,...

thomas-nilsson-irfu commented 3 years ago

After some feedback from @ahmadlalti, it appears that this issue is also impacting recent MacOS installations. As the April 2021 is fast approaching (when Ubuntu 16.04 reaches its "end of standard support") all Ubuntu users should have updated to at least release 18.04 and this issue can be fixed by re-compiling the mex files.

For this I would prefer some feedback from the general irfu-matlab community on which MacOS version I should use for this compiling (it may be that if I compile with the latest most recent version of MacOS it is not compatible with older versions). Please feel free to comment below, if no comments are given I suspect I may try to get hold of a MacOS 10.14 machine (which according to wikipedia is the oldest still supported version as of time of writing).

(Note to self: remember the issue with 55cb55fc171f936864328a3ec78de6f4548c0aa1.)

sergiotr commented 3 years ago

Thanks Thomas for the update. I currently use MacOS 10.13 and the current version of whamp works fine. I wonder if various compilations, for various versions, can be kept in the irfu_matlab package somehow? Thanks for all the support, irfu-matlab is just great!

El mar, 9 mar 2021 a las 10:14, Thomas Nilsson (notifications@github.com) escribió:

After some feedback from @ahmadlalti https://github.com/ahmadlalti, it appears that this issue is also impacting recent MacOS installations. As the April 2021 is fast approaching (when Ubuntu 16.04 reaches its "end of standard support") all Ubuntu users should have updated to at least release 18.04 and this issue can be fixed by re-compiling the mex files.

For this I would prefer some feedback from the general irfu-matlab community on which MacOS version I should use for this compiling (it may be that if I compile with the latest most recent version of MacOS it is not compatible with older versions). Please feel free to comment below, if no comments are given I suspect I may try to get hold of a MacOS 10.14 machine (which according to wikipedia is the oldest still supported version as of time of writing).

(Note to self: remember the issue with 55cb55f https://github.com/irfu/irfu-matlab/commit/55cb55fc171f936864328a3ec78de6f4548c0aa1 .)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/irfu/irfu-matlab/issues/61#issuecomment-793596719, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADB7KOPPHN2DUHCHHHKLQNDTCXRIJANCNFSM4TJ4LI5Q .

ajohlander commented 3 years ago

I can add that I'm using Mac OS 10.14 and WHAMP works fine when I'm using Matlab 2016b. However, I get these same errors when I'm using a newer version of Matlab, like 2019a. I have not tried to figure out why.

ahmadlalti commented 3 years ago

Following the advice of Louis I installed an older version of gfortran and it works. so in case anyone faced similar problem these are the steps given by Louis: install Homebrew using the following command in terminal /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" give Homebrew access to read and write using: sudo chown -R $(whoami) $(brew --prefix)/* Finally install an older version of gfortran (in gcc 4.9) brew install gcc49

nykyrik commented 3 years ago

Hi All,

I also had this problem when using macOS Big Sur (11.2.1)with MATLAB R2020b and the latest Bigsur version of gfortran (after compilation the format of arrays was not recognized by mex)

the above trick (downloading the older version gcc4.9 with Homebrew) gave me first the following error MEX configured to use 'gfortran' for FORTRAN language compilation.

mex libwhamp.a mexwhamp.F Building with 'gfortran'. Error using mex /usr/local/Cellar/gcc@4.9/4.9.4_2/bin/gfortrani8: No such file or directory

I was able to fix this by copying /usr/local/Cellar/gcc/10.2.0_4/bin/gfortrani8 into /usr/local/Cellar/gcc@4.9/4.9.4_2/bin/

seems to work now. Thanks Daniel Graham for pointing me to this thread!!! Instead of copying it to this folder one could also probably get it to work directly pointing to /usr/local/Cellar/gcc/10.2.0_4/bin/

mex libwhamp.a mexwhamp.F Building with 'gfortran'. MEX completed successfully.

nykyrik commented 3 years ago

I guess i celebrated too early: now when running Output = whamp.run([],[]) with the new mexwhamp.mexmaci64 in +whampfolder it starts running but then crashes the Matlab. I think this is related to compiler linking problem. "was built for newer macOS version (11.0) than being linked (10.16.3)"

nykyrik commented 3 years ago

I got the matlab version of WHAMP working now in Big Sur with Matlab2020b and xcode12.4. I switched to newer version of gfortran/Cellar/gcc/10.2.0_4/bin/gfortran and copied gfortrani8 settings into this folder and hacked the SDKs with correct Mac version.

Interestingly now the mex libwhamp.a mexwhamp.F

gives an error now,

but I am able to run whamp from matlab with the old mexwhamp-file.

Stranger things.

thomas-nilsson-irfu commented 3 years ago

I'm closing this issue here following that it has been updated for Linux 64b systems, which was included in the new irfu-matlab release v1.16.0.

irfu / irfu-matlab