gcc error when Make

[alanxuhaojie]

General information

• Corrfunc version:2.3.4
• platform: Linux x86
• installation method (pip/source/other?):

Hey, Corrfunc is used to be successfully installed with python 3.7 but it failed after I upgrade to python 3.8. I checked my gcc version is 4.8.5, and the version of make, gsl, python, and numpy seem all good. I google a bit this issue and some say upgrade gcc would solve the problem and others suggest to change the line of CFLAGS. Any ideas? Thanks

Issue description

mkdir -p lib make -C theory

-------COMPILE SETTINGS------------ MAKE = ["make"] CC = ["gcc"] OPT = ["-DPERIODIC -DENABLE_MIN_SEP_OPT -DCOPY_PARTICLES -DUSE_OMP"] CFLAGS = ["-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/hjxu/anaconda3/include -DVERSION=\"2.3.4\" -DUSE_UNICODE -std=c99 -m64 -g -Wsign-compare -Wall -Wextra -Wshadow -Wunused -fPIC -D_POSIX_SOURCE=200809L -D_GNU_SOURCE -D_DARWIN_C_SOURCE -O3 -ftree-vectorize -funroll-loops -fprefetch-loop-arrays --param simultaneous-prefetches=4 -fopenmp -funroll-loops -march=native -fno-strict-aliasing -Wformat=2 -Wpacked -Wnested-externs -Wpointer-arith -Wredundant-decls -Wfloat-equal -Wcast-qual -Wcast-align -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wstrict-prototypes -Wno-unused-local-typedefs "] CLINK = [" -lrt -fopenmp -lm"] PYTHON = ["python"] GSL_CFLAGS = ["-I/home/hjxu/anaconda3/include"] GSL_LINK = ["-L/home/hjxu/anaconda3/lib -lgsl -lgslcblas -lm -Xlinker -rpath -Xlinker /home/hjxu/anaconda3/lib"] PYTHON_CFLAGS = ["-isystem/home/hjxu/anaconda3/include/python3.8 -isystem /home/hjxu/anaconda3/lib/python3.8/site-packages/numpy/core/include/numpy/"] -------END OF COMPILE SETTINGS------------

make[1]: 进入目录“/home/hjxu/bin/Corrfunc/theory” make -C DD make[2]: 进入目录“/home/hjxu/bin/Corrfunc/theory/DD” gcc -DPERIODIC -DENABLE_MIN_SEP_OPT -DCOPY_PARTICLES -DUSE_OMP -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/hjxu/anaconda3/include -DVERSION=\"2.3.4\" -DUSE_UNICODE -std=c99 -m64 -g -Wsign-compare -Wall -Wextra -Wshadow -Wunused -fPIC -D_POSIX_SOURCE=200809L -D_GNU_SOURCE -D_DARWIN_C_SOURCE -O3 -ftree-vectorize -funroll-loops -fprefetch-loop-arrays --param simultaneous-prefetches=4 -fopenmp -funroll-loops -march=native -fno-strict-aliasing -Wformat=2 -Wpacked -Wnested-externs -Wpointer-arith -Wredundant-decls -Wfloat-equal -Wcast-qual -Wcast-align -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wstrict-prototypes -Wno-unused-local-typedefs -DVERSION=\"2.3.4\" -DUSE_UNICODE -std=c99 -m64 -g -Wsign-compare -Wall -Wextra -Wshadow -Wunused -fPIC -D_POSIX_SOURCE=200809L -D_GNU_SOURCE -D_DARWIN_C_SOURCE -O3 -ftree-vectorize -funroll-loops -fprefetch-loop-arrays --param simultaneous-prefetches=4 -fopenmp -funroll-loops -march=native -fno-strict-aliasing -Wformat=2 -Wpacked -Wnested-externs -Wpointer-arith -Wredundant-decls -Wfloat-equal -Wcast-qual -Wcast-align -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wstrict-prototypes -Wno-unused-local-typedefs -I../../io -I../../utils -c DD.c -o DD.o gcc: 错误：unrecognized command line option ‘-fno-plt’ make[2]: [DD.o] 错误 1 make[2]: 离开目录“/home/hjxu/bin/Corrfunc/theory/DD” make[1]: [DD] 错误 2 make[1]: 离开目录“/home/hjxu/bin/Corrfunc/theory” make: *** [theory] 错误 2

[lgarrison]

How are you installing Corrfunc? From source with make, or pip/conda install? The -fno-plt option that is causing the error isn't put there by Corrfunc; is it possible that the CFLAGS environment variable has been set elsewhere, perhaps by a module command, for instance? Corrfunc will respect any environment CFLAGS, but they have to be compatible with gcc.

As a quick fix, you might be able to tell Corrfunc not to use the environment CFLAGS by changing this line to read CFLAGS :=. Or you can upgrade gcc, especially if you're on a module system.

If this is a conda install, the problem might be that the build is not finding the conda compiler, but CFLAGS are set as if it is. That would need some more thought (possibly installing from source would fix it).

[alanxuhaojie]

Thanks for the quick respond. I install Corrfunc from source with make.

The quick fix by changing the CFLAGS line works. I have two further questions: 1, Will the CFLAGS line changing thing downgrade the performance or anything? 2, when 'make tests', it shows the warnings go WARNING: File was written in a different precision than requested (file precision = 4 requested precision = 8) [Warning] The CPU supports AVX512F but the compiler does not. Can you try another compiler? Does these two warnings matter?

[lgarrison]

I just did some tests, and that CFLAGS prefix seems to be coming in because gcc_linux-64 is installed in the conda environment. The CC variable is set to point to the conda compiler that works with those CFLAGS, but Corrfunc is ignoring CC because it sometimes points to an outdated cc compiler. So it kind of is our fault!

Your current installation should work fine, although the warning about AVX512F means that you probably won't be getting maximum performance. You could try restoring the CFLAGS line, and instead comment out this line. If you make distclean; make, then hopefully it will be able to compile with the conda compiler and use AVX512. The file precision warning is safe to ignore.

[manodeep]

Yup - Corrfunc will ignore the environment CC by default. However, you can invoke the right compiler by invoking make CC=<customCC>. Or, in the case of the python setup, using python setup.py install <--user> CC=customCC

@alanxuhaojie Depending on the problem setup, running the AVX512F kernel can be a significant speed boost. But the AVX512 support only landed in more recent versions of gcc - you need at least gcc7.3 to correctly compile Corrfunc with AVX512

[alanxuhaojie]

@lgarrison As Manodeep points out, the AVX512 support may need more recent versions of gcc and I do not want to screw other dependency :( so I'm fine with my current installation. Big thanks to you guys for very quick and useful response. I will close with issue.

[lgarrison]

I do think that the Anaconda compiler would support AVX512, but it's not a given that Corrfunc would find all of its dependencies under that toolchain! For what it's worth, I got it to work locally, though. Happy to help if you decide you want the extra performance.

[manodeep]

Not sure about gcc4.8.5 but it should be possible to install a gcc version that supports AVX512. Seems that gcc7 is installed in the gcc_linux-64 package (see here) but the conda compiler has a long name -- x86_64-conda_cos6-linux-gnu-gcc.

@alanxuhaojie What happens if you do the following:

source activate <conda_env_with_gcc_linux-64> 
make distclean 
make CC=x86_64-conda_cos6-linux-gnu-gcc

[manodeep]

And I will second @lgarrison's offer to figure out a solution - no need to run with sub-par performance!

[manodeep]

Does installing binutils from conda-forge (conda install -c conda-forge binutils) work?

[alanxuhaojie]

@manodeep @lgarrison Thanks, let's figure out a solution. I simply comment out :cc line and make distclean, then make, as suggested. It finds the right complier and but it reports a binutils problem, as showed below, even after I successfully installed binutils 2.32 after the command (conda install -c conda-forge binutils). Should I change something in common.mk to help Corrfunc to identify the new binutils?

If you want to permanently set the default compiler to /home/hjxu/anaconda3/bin/x86_64-conda_cos6-linux-gnu-cc for all future compilations, please update the "CC" variable in "common.mk" make[1]: 进入目录“/home/hjxu/bin/Corrfunc/theory” ../common.mk:371: DISABLING AVX-512 SUPPORT DUE TO GNU ASSEMBLER BUG. UPGRADE TO BINUTILS >=2.32 TO FIX THIS.

[manodeep]

Huh - that's odd. The conda-forge installed binutils should be v2.35 - seems like that is not being picked up. What do you get when you do which ld and ld -v? And can you locate the (newly installed) ld under the relevant conda (env)-directory?

[alanxuhaojie]

It does fails two times when conda install -c conda-forge binutils but it tried another jason file-ish? and succeed.

(base) [hjxu@dt640 Corrfunc]$ which ld ~/anaconda3/bin/ld (base) [hjxu@dt640 Corrfunc]$ ld -v GNU ld (GNU Binutils) 2.32

[lgarrison]

It should know how to find its own binutils (and does on my system). What happens if you run $CC $CFLAGS -xc -Wa,-v -c /dev/null -o /dev/null 2>&1? That should show the assembler it's using.

[alanxuhaojie]

(base) [hjxu@dt640 Corrfunc]$ $CC $CFLAGS -xc -Wa,-v -c /dev/null -o /dev/null 2>&1 GNU assembler version 2.31.1 (x86_64-conda_cos6-linux-gnu) using BFD version (crosstool-NG 1.23.0.444-4ea7) 2.31.1

Weird that it shows different assembler 2.31 with that from ld -v

[lgarrison]

Ah, you're on base Anaconda, not conda-forge. Base Anaconda is still using the bugged assembler. Try: conda install -c conda-forge gcc_linux-64, then try rebuilding Corrfunc.

On Tue, Mar 2, 2021 at 8:09 PM Haojie Xu notifications@github.com wrote:

(base) [hjxu@dt640 Corrfunc]$ $CC $CFLAGS -xc -Wa,-v -c /dev/null -o /dev/null 2>&1 GNU assembler version 2.31.1 (x86_64-conda_cos6-linux-gnu) using BFD version (crosstool-NG 1.23.0.444-4ea7) 2.31.1

Weird that it shows different assembler 2.31 with that from ld -v

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/manodeep/Corrfunc/issues/243#issuecomment-789346559, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLA7SZWGVDDD5M3NVBMHRTTBWD4FANCNFSM4YMDQQ7Q .

-- Lehman Garrison Flatiron Research Fellow, Cosmology X Data Science Group Center for Computational Astrophysics, Flatiron Institute lgarrison.github.io

[alanxuhaojie]

Still no luck! The environmental staff are tough to understand through...

Tried conda install -c conda-forge gcc_linux-64, then make distclean and then make.

[lgarrison]

Hmm, and the assembler version is still 2.31.1, not 2.35.1?

On Tue, Mar 2, 2021 at 9:06 PM Haojie Xu notifications@github.com wrote:

Still no luck! The environmental staff are tough to understand through...

Tried conda install -c conda-forge gcc_linux-64, then make distclean and then make.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/manodeep/Corrfunc/issues/243#issuecomment-789369102, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLA7S2CXXCZOCNRVTXJUFDTBWKSJANCNFSM4YMDQQ7Q .

-- Lehman Garrison Flatiron Research Fellow, Cosmology X Data Science Group Center for Computational Astrophysics, Flatiron Institute lgarrison.github.io

[alanxuhaojie]

Yup, it reports the same problem when make.

(base) [hjxu@dt640 Corrfunc]$ $CC $CFLAGS -xc -Wa,-v -c /dev/null -o /dev/null 2>&1 GNU assembler version 2.31.1 (x86_64-conda_cos6-linux-gnu) using BFD version (crosstool-NG 1.23.0.444-4ea7) 2.31.1

[manodeep]

To isolate what is going on, it might be better to create a new environment and then work within that environment.

@alanxuhaojie Can you try the following:

conda create -n corrfunc python=3.8 numpy=1.18
conda/source activate corrfunc
conda install -c conda-forge gcc_linux-64 gsl binutils
cd </path/to/Corrfunc>
make distclean
make CC=x86_64-conda_cos6-linux-gnu-gcc

[alanxuhaojie]

Hi Manodeep, it works! I do see a significant performance for make tests using AVX512 kernel. Btw, when I type $CC $CFLAGS -xc -Wa,-v -c /dev/null -o /dev/null 2>&1, it shows GNU assembler version 2.35.1 (x86_64-conda-linux-gnu) using BFD version (GNU Binutils) 2.35.1. What if I want to install Corrfunc on my base environment, what I can do?

[lgarrison]

Great! Maybe try including binutils_linux-64 in the install command in the base environment? I.e. conda install -c conda-forge gcc_linux-64 binutils_linux-64? And after running that, try running conda list and make sure you see:

binutils_impl_linux-64 2.35.1 h193b22a_2 conda-forge binutils_linux-64 2.35 h67ddf6f_30 conda-forge

On Tue, Mar 2, 2021 at 11:02 PM Haojie Xu notifications@github.com wrote:

Hi Manodeep, it works! I do see a significant performance for make tests using AVX512 kernel. Btw, when I type $CC $CFLAGS -xc -Wa,-v -c /dev/null -o /dev/null 2>&1, it shows GNU assembler version 2.35.1 (x86_64-conda-linux-gnu) using BFD version (GNU Binutils) 2.35.1. What if I want to install Corrfunc on my base environment, what I can do?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/manodeep/Corrfunc/issues/243#issuecomment-789410491, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLA7SYALDUVJQBFL6DJYDLTBWYD5ANCNFSM4YMDQQ7Q .

-- Lehman Garrison Flatiron Research Fellow, Cosmology X Data Science Group Center for Computational Astrophysics, Flatiron Institute lgarrison.github.io

[alanxuhaojie]

Still nope :(

binutils_impl_linux-64 2.31.1 h6176602_1 binutils_linux-64 2.31.1 h6176602_9

[lgarrison]

It should tell you the channel that those packages are from. It should be conda-forge; if not, then perhaps the primary channel has some kind of precedence. @manodeep?

On Tue, Mar 2, 2021 at 11:25 PM Haojie Xu notifications@github.com wrote:

Still nope :(

binutils_impl_linux-64 2.31.1 h6176602_1 binutils_linux-64 2.31.1 h6176602_9

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/manodeep/Corrfunc/issues/243#issuecomment-789418027, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLA7S6ONVGUGDG3M3SMRDTTBW23VANCNFSM4YMDQQ7Q .

-- Lehman Garrison Flatiron Research Fellow, Cosmology X Data Science Group Center for Computational Astrophysics, Flatiron Institute lgarrison.github.io

[manodeep]

May be first uninstall those two packages and then re-install from conda-forge?

[alanxuhaojie]

After I uninstalled these two packages, and reinstall them by 'conda install -c conda-forge gcc_linux-64 binutils_linux-64', it shows the below, the channel seems incorrect. They should be conda-force rather than main, right? After the install, checked 'conda list', still 2.31

(base) [hjxu@dt640 Corrfunc]$ conda install -c conda-forge gcc_linux-64 binutils_linux-64 Collecting package metadata (current_repodata.json): done Solving environment: done

Package Plan

environment location: /home/hjxu/anaconda3

added / updated specs:

• binutils_linux-64
• gcc_linux-64

The following NEW packages will be INSTALLED:

binutils_impl_lin~ pkgs/main/linux-64::binutils_impl_linux-64-2.31.1-h6176602_1 binutils_linux-64 pkgs/main/linux-64::binutils_linux-64-2.31.1-h6176602_9 gcc_impl_linux-64 pkgs/main/linux-64::gcc_impl_linux-64-7.3.0-habb00fd_1 gcc_linux-64 pkgs/main/linux-64::gcc_linux-64-7.3.0-h553295d_9

The following packages will be SUPERSEDED by a higher-priority channel:

conda pkgs/main::conda-4.9.2-py38h06a4308_0 --> conda-forge::conda-4.9.2-py38h578d9bd_0

[lgarrison]

This may be some combination of the base environment being special, and/or channel priority. base is where conda itself is installed, so maybe the main channel has special priority there. You can set conda-forge as higher priority by default with:

conda config --add channels conda-forge
conda config --set channel_priority strict

(from https://conda-forge.org/docs/user/introduction.html#how-can-i-install-packages-from-conda-forge), but I would only recommend that if you want to switch all your environments over to conda-forge. Otherwise, I would use a separate environment and run those commands, but with the --env flag to tell conda to apply the change just to the active env.

[lgarrison]

And you may want to update conda itself, since package resolution rules sometimes change between versions: conda update conda

[alanxuhaojie]

@lgarrison Hi Lehman, you are right on the base environment being special. I simply clone my base env to a new env and install Corrfunc successfully by following the steps from @manodeep

conda activate newenv conda install -c conda-forge gcc_linux-64 gsl binutils cd </path/to/Corrfunc> make distclean make CC=x86_64-conda_cos6-linux-gnu-gcc

So, to sum it up, if one is in Anaconda base env, he/she won't successfully install Corrfunc correctly due to the bugged assembler. The only way (so far I know) is to set up a new environment so that one can install worked version assembler by conda install -c conda-forge gcc_linux-64 gsl binutils and then install Corrfunc in that environment. For the new environment, you can simply clone your current environment so that you have all the installed package. That's pretty much I can conclude. @manodeep @lgarrison Thanks for your time and effort. I really appreciate it.

[lgarrison]

Great, glad that worked. Hopefully conda will start using a newer version of binutils before too long, which will make this easier.

[manodeep]

Yup - glad we could sort out the issue. Hope the better performance helps with your project