search

jfowkes / pycutest

Python interface to CUTEst
https://jfowkes.github.io/pycutest/
GNU General Public License v3.0
28 stars 11 forks source link

Segmentation fault: 11 #9

Closed ZiyanZhu1994 closed 2 years ago

ZiyanZhu1994 commented 5 years ago

Hi,

I updated my Mac OS system several days ago. When I run the PyCUTEst today, it returns some warnings and a Segmentation fault:

ld: warning: directory not found for option '-L/usr/local/gfortran/lib/' ld: warning: object file (RANGE.o) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (EXTER.o) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (ELFUN.o) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (GROUP.o) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cdh.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cdimen.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cdimsh.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cdimsj.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cfn.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cidh.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ccfg.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ccfsg.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ccifg.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ccifsg.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cgr.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cgrdh.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(chprod.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cjprod.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cofg.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(csetup.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(csgr.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(csgrsh.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(uhprod.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(uofg.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cish.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(connames.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(creport.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(csh.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cvartype.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(probname.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(udh.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(udimsh.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ugrdh.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ugrsh.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ureport.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(usetup.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ush.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(uvartype.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(varnames.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(fortran_ops.o)) was built for newer OSX version (10.14) than being linked (10.9) ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cutest.o)) was built for newer OSX version (10.14) than being linked (10.9) Segmentation fault: 11

Do you know why it happens and how to fix it? Many thanks!

jfowkes commented 5 years ago

Hmm... could you reinstall CUTEst then clear out the PyCUTEst cache and try again? I suspect that this is an issue with incompatible compiler versions.

ZiyanZhu1994 commented 5 years ago

I have reinstalled the CUTEst using the brew reinstall. When I run the PyCUTEst in the terminal, the same warnings are returned, but it can work. However, when I try to run PyCUTEst in Jupyter notebook or other IDEs, the kernel will be died and Segmentation fault error will occur.

jfowkes commented 5 years ago

Could you please install the latest version of Command Line Tools using xcode-select --install as described here: http://railsapps.github.io/xcode-command-line-tools.html? Then clear out the PyCUTEst cache and try again.

ZiyanZhu1994 commented 5 years ago

I reinstalled the Xcode and clear out the cache. It still returns these warnings. Below is the version of gcc: Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1 Apple LLVM version 10.0.1 (clang-1001.0.46.4) Target: x86_64-apple-darwin18.7.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin I think it is the latest version. The version of gfortran is: GNU Fortran (Homebrew GCC 9.2.0) 9.2.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

jfowkes commented 5 years ago

Yes that's exactly the same as I have. You're using Anaconda's python right? Have you also updated Anaconda to the latest version for you newly upgraded Mac OS system?

ZiyanZhu1994 commented 5 years ago

Yes, I have updated the anaconda. Below is the information of my anaconda: active environment : base active env location : //anaconda3 shell level : 1 user config file : /Users/ziyan/.condarc populated config files : /Users/ziyan/.condarc conda version : 4.7.11 conda-build version : 3.18.9 python version : 3.7.4.final.0 virtual packages : base environment : //anaconda3 (writable) channel URLs : https://repo.anaconda.com/pkgs/main/osx-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/osx-64 https://repo.anaconda.com/pkgs/r/noarch package cache : //anaconda3/pkgs /Users/ziyan/.conda/pkgs envs directories : //anaconda3/envs /Users/ziyan/.conda/envs platform : osx-64 user-agent : conda/4.7.11 requests/2.22.0 CPython/3.7.4 Darwin/18.7.0 OSX/10.14.6 UID:GID : 501:20 netrc file : None offline mode : False

By the way, my Mac OS version is Mojave 10.14.6.

jfowkes commented 5 years ago

Okay so I installed Anaconda and have been able to reproduce and fix your error. The issue is that for some stupid reason Anaconda sets the environment variable MACOSX_DEPLOYMENT_TARGET=10.9 by default (yes that's right, MacOS Mavericks!). So somewhere you have to set MACOSX_DEPLOYMENT_TARGET=10.14 either by doing export MACOSX_DEPLOYMENT_TARGET=10.14 in the terminal or in .bashrc. If you're using an IDE, you'll have to set it in the appropriate place for setting environment variables in your IDE (see the documentation for your IDE). There may well be a way of changing this stupid default somewhere in Anaconda itself, but I'm not familiar enough with Anaconda to know where that would be and I haven't been able to find anything.

ZiyanZhu1994 commented 5 years ago

Thanks! The linking warnings are fixed! There is still one warning: ld: warning: directory not found for option '-L/usr/local/gfortran/lib/' and Segmentation fault:11. I have tried to reinstall the gcc and cutest using brew, but the error is still there.

jfowkes commented 5 years ago

You can ignore the option '-L/usr/local/gfortran/lib/' warning, that's just an additional gfortran directory we added that we should probably remove. That's strange, it works for me: Screenshot

What is your gfortran --version? Are you using the one provided by Anaconda? I am not.

ZiyanZhu1994 commented 5 years ago

Yes, it is strange. It used to work well before I updating MacOS from 10.14.5 to 10.14.6. gfortran was installed using brew along with gcc. Here is the version of it: GNU Fortran (Homebrew GCC 9.2.0) 9.2.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Also, I run cd ~/Documents/pycutest_cache/pycutest_cache_holder/ROSENBR/ rm -rf build/ DISTUTILS_DEBUG=1 python setup.py build it returns

options (after parsing config files): options (after parsing command line): option dict for 'build' command: {} running build Distribution.get_command_obj(): creating 'build' command object running build_ext Distribution.get_command_obj(): creating 'build_ext' command object building '_pycutestitf' extension creating build creating build/temp.macosx-10.9-x86_64-3.7 gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I//anaconda3/include -arch x86_64 -I//anaconda3/include -arch x86_64 -DLINUX -I//anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy -I//anaconda3/include/python3.7m -c cutestitf.c -o build/temp.macosx-10.9-x86_64-3.7/cutestitf.o creating build/lib.macosx-10.9-x86_64-3.7 gcc -bundle -undefined dynamic_lookup -L//anaconda3/lib -arch x86_64 -L//anaconda3/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.9-x86_64-3.7/cutestitf.o RANGE.o EXTER.o ELFUN.o GROUP.o /usr/local/opt/cutest/libexec/objects/mac64.osx.gfo/double/libcutest.a -L/usr/local/Cellar/gcc/9.2.0/lib/gcc/9/ -L/usr/local/gfortran/lib/ -lgfortran -o build/lib.macosx-10.9-x86_64-3.7/_pycutestitf.cpython-37m-darwin.so -Wl,-no_compact_unwind ld: warning: directory not found for option '-L/usr/local/gfortran/lib/'

jfowkes commented 5 years ago

Yes that's the same gcc version as I have. According to your output, you're still building for MacOS 10.9 though, you should be seeing creating build/temp.macosx-10.14-x86_64-3.7, could you try export MACOSX_DEPLOYMENT_TARGET=10.14 and then DISTUTILS_DEBUG=1 python setup.py build and post the output? I can then compare it to mine and we should be able to figure out what's going on.

ZiyanZhu1994 commented 5 years ago

I have tried export MACOSX_DEPLOYMENT_TARGET=10.14 in the terminal. It still uses MacOS 10.9. However, the issue is solved by creating another python environment using conda, where only PyCUTEst is installed. I guess there might be come conflicts between some packages in anaconda and PyCUTEst.

jfowkes commented 5 years ago

Yes I think that's probably the case, I was discussing this issue with one of the Anaconda developers yesterday and they told me that the environments target 10.9 for compatibility and anything else is only supported on a best-effort basis as they're largely a compile it for you distribution (unlike homebrew).

ladsjohn commented 3 years ago

Curious how you "solved by creating another python environment using conda, where only PyCUTEst is installed". I tried: $ conda create -n hansolo $ conda activate hansolo

I'm still hitting a Seg Fault:11 when trying to run: problem = pycutest.import_problem('ARGLALE', sifParams={'N':100, 'M':200}) print(problem)

jfowkes commented 3 years ago

The issue here is that Anaconda targets a very old version of MacOS when compiling and you need to tell it to target your current MacOS version (which should be the default but Anaconda developers don't want to change this). You can do this by setting export MACOSX_DEPLOYMENT_TARGET=10.14 (or whatever your current MacOS version is, 10.14=Mojave, 10.15=Catalina, etc) in the terminal before running Anaconda python. You may need to create a new python environment using conda where only PyCUTEst is installed in order for it to work.

In general I would discourage you from using Anaconda with PyCUTEst as it is a compile for you distribution and support for compiling things is experimental as the Anaconda developers made very clear when I raised this issue with them. My recommendation would be to install python via homebrew and use that instead.

ladsjohn commented 3 years ago

I had previously tried setting MACOSX_DEPLOYMENT_TARGET=10.15 in my environment to no avail. On your recommendation; I did have success by not using Anaconda. Many thanks!