search

OpenMathLib / OpenBLAS

OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version.
http://www.openblas.net
BSD 3-Clause "New" or "Revised" License
6.36k stars 1.49k forks source link

cc1: error: CPU you selected does not support x86-64 instruction set #4541

Closed yejianbang closed 7 months ago

yejianbang commented 7 months ago

I extracted the source code of openblas v0.3.18 Then I went into the source code directory. The following error is reported when the "make" command is executed: cc1: error: CPU you selected does not support x86-64 instruction set

My environment information is as follows: gcc 11.3.0 arch x86_64 cpu AMD

Also, the strange thing is that I use gcc7 will compile successfully, why is that?

Looking forward to the reply, thanks！

martin-frbg commented 7 months ago

It is possible that 0.3.18 does not identify your cpu correctly, is there a reason why you want to use this release from two years ago ? Depending on the model of your AMD cpu, you could try building with "make TARGET=ZEN" or "make TARGET=SKYLAKEX" to avoid the autodetection. (The newer gcc may have more checks, or may include newer header files for SIMD intrinsics)

yejianbang commented 7 months ago

@martin-frbg Thank you for your reply. I'm trying the latest version 0.3.26 and use the command of "make TARGET=ZEN" or "make TARGET=SKYLAKEX" , Unfortunately, I also encountered the same error.

Does this seem to have something to do with the CPU architecture or the GCC version? Thanks for replying again.

martin-frbg commented 7 months ago

Strange - could you let me know what your cpu is, please ? Are you running this in a virtual machine, where maybe the cpu information is not fully passed on and "looks like" 32bit to the operating system ? Or maybe you installed a 32bit version of gcc11 by mistake ?

yejianbang commented 7 months ago

@martin-frbg Thank you for your reply. My CPU model name is “Hygon C86 7390 32-core Processor”, I'm building openblas on a physical machine.

My gcc related information is as follows: Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/workspace/software/compiler/gcc/11.3.0/libexec/gcc/x86_64-pc-linux-gnu/11.3.0/lto-wrapper Target: x86_64-pc-linux-gnu Configured with: ./configure --disable-multilib --enable-languages=c,c++,fortran --prefix=/workspace/software/compiler/gcc/11.3.0 --disable-static --enable-shared Thread model: posix Supported LTO compression algorithms: zlib gcc version 11.3.0 (GCC)

My OS related information is as follows: NAME="Kylin Linux Advanced Server" VERSION="V10 (Tercel)" ID="kylin" VERSION_ID="V10" PRETTY_NAME="Kylin Linux Advanced Server V10 (Tercel)" ANSI_COLOR="0;31"

I ran the command "getconf LONG_BIT" and it returned "64"

martin-frbg commented 7 months ago

Very strange. OpenBLAS does not have as detailed cpu identification lists for Hygon as for Intel or AMD models, but if the 7390 belongs to the original Dhyana family (CPUID family code 18h) it should be autodetected as ZEN already, and I do not see anything in the build system code that would make it treat the cpu as 32bit (unless the C compiler claimed it was 32bit by defining __i386 or _X86 instead of __x86_64__ or __amd64__). Do you get the compiler error immediately when you start make, or does it build some files before it gives up ? (Do you have the Makefile.conf and config.h that are generated by getarch, or is it already failing to build getarch itself ?) Could this be something like a forgotten -m32 in your default CFLAGS environment variable ??

yejianbang commented 7 months ago

Yes, I manually went to the code to generate the "getarch" binary, and when I did “getarch 1 > makefile.conf”, it should be autodetected as ZEN already.

If I do a "make", it doesn't generate "Makefile.conf" and "config.h" and "getarch ", It returned the following error: [root@localhost OpenBLAS-0.3.18]# make cc1: error: CPU you selected does not support x86-64 instruction set cc1: error: CPU you selected does not support x86-64 instruction set make: [Makefile.prebuild:70: getarch] Error 1 Makefile.system:1397: Makefile.: No such file or directory make: No rule to make target 'Makefile.'. Stop. [root@localhost OpenBLAS-0.3.18]#

I don't think I have any extra CFLAGS set up in my environment.

martin-frbg commented 7 months ago

Hmm. If you can compile the getarch utility without problems, but the build still fails very early, the only other spot in the code I can think of is the c_check script which compiles a few simple C files to find out compiler capabilities. Can you run this as ./c_check conf header gcc (with "gcc" being the full path to your gcc-11 if necessary) ? It should produce two files "conf" and "header" which are the first parts of the Makefile.conf and config.h - the "conf" file should contain the line BINARY64=1 if the compiler was found to support 64bit.

yejianbang commented 7 months ago

Yes, I run ". /c_check conf header gcc" and produce two files "conf" and "header"

conf: OSNAME=Linux ARCH=x86_64 C_COMPILER=GCC BINARY32= BINARY64=1 CEXTRALIB=-L/workspace/software/compiler/gcc/11.3.0/lib/gcc/x86_64-pc-linux-gnu/11.3.0 -L/workspace/software/compiler/gcc/11.3.0/lib/gcc/x86_64-pc-linux-gnu/11.3.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/workspace/software/compiler/gcc/11.3.0/lib/gcc/x86_64-pc-linux-gnu/11.3.0/../../.. -lc

header:

define OS_LINUX 1

define ARCH_X86_64 1

define C_GCC 1

define 64BIT 1

define HAVE_C11 1

define PTHREAD_CREATE_FUNC pthread_create

I think it's 64-bit support.

martin-frbg commented 7 months ago

Strange, that looks perfectly normal. I do not think Makefile.system could add something bad to the GETARCH_CFLAGS used for building getarch, but could you try running make CC="gcc -v" getarch >log 2>&1 to see what ends up in the gcc command line ?

yejianbang commented 7 months ago

I run the command " running make CC="gcc -v" getarch >log 2>&1", the “log” is:

Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/workspace/software/compiler/gcc/11.3.0/libexec/gcc/x86_64-pc-linux-gnu/11.3.0/lto-wrapper Target: x86_64-pc-linux-gnu Configured with: ./configure --disable-multilib --enable-languages=c,c++,fortran --prefix=/workspace/software/compiler/gcc/11.3.0 --disable-static --enable-shared Thread model: posix Supported LTO compression algorithms: zlib gcc version 11.3.0 (GCC) COLLECT_GCC_OPTIONS='-v' '-march=native' '-D' 'GEMM_MULTITHREAD_THRESHOLD=4' '-D' 'NO_PARALLEL_MAKE=0' '-o' 'getarch' '-dumpdir' 'getarch-' /workspace/software/compiler/gcc/11.3.0/libexec/gcc/x86_64-pc-linux-gnu/11.3.0/cc1 -quiet -v -D GEMM_MULTITHREAD_THRESHOLD=4 -D NO_PARALLEL_MAKE=0 getarch.c -march=i386 -mno-mmx -mno-popcnt -mno-sse -mno-sse2 -mno-sse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -mno-avx -mno-avx2 -mno-sse4a -mno-fma4 -mno-xop -mno-fma -mno-avx512f -mno-bmi -mno-bmi2 -mno-aes -mno-pclmul -mno-avx512vl -mno-avx512bw -mno-avx512dq -mno-avx512cd -mno-avx512er -mno-avx512pf -mno-avx512vbmi -mno-avx512ifma -mno-avx5124vnniw -mno-avx5124fmaps -mno-avx512vpopcntdq -mno-avx512vbmi2 -mno-gfni -mno-vpclmulqdq -mno-avx512vnni -mno-avx512bitalg -mno-avx512bf16 -mno-avx512vp2intersect -mno-3dnow -mno-adx -mno-abm -mno-cldemote -mno-clflushopt -mno-clwb -mno-clzero -mno-cx16 -mno-enqcmd -mno-f16c -mno-fsgsbase -mno-fxsr -mno-hle -mno-sahf -mno-lwp -mno-lzcnt -mno-movbe -mno-movdir64b -mno-movdiri -mno-mwaitx -mno-pconfig -mno-pku -mno-prefetchwt1 -mno-prfchw -mno-ptwrite -mno-rdpid -mno-rdrnd -mno-rdseed -mno-rtm -mno-serialize -mno-sgx -mno-sha -mno-shstk -mno-tbm -mno-tsxldtrk -mno-vaes -mno-waitpkg -mno-wbnoinvd -mno-xsave -mno-xsavec -mno-xsaveopt -mno-xsaves -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -mno-uintr -mno-hreset -mno-kl -mno-widekl -mno-avxvnni -mtune=generic -quiet -dumpdir getarch- -dumpbase getarch.c -dumpbase-ext .c -version -o /tmp/ccVep3kP.s GNU C17 (GCC) version 11.3.0 (x86_64-pc-linux-gnu) compiled by GNU C version 11.3.0, GMP version 6.1.0, MPFR version 3.1.6, MPC version 1.0.3, isl version isl-0.18-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 ignoring nonexistent directory "/workspace/software/compiler/gcc/11.3.0/lib/gcc/x86_64-pc-linux-gnu/11.3.0/../../../../x86_64-pc-linux-gnu/include"

include "..." search starts here:

include <...> search starts here:

/workspace/software/compiler/gcc/11.3.0/lib/gcc/x86_64-pc-linux-gnu/11.3.0/include /usr/local/include /workspace/software/compiler/gcc/11.3.0/include /workspace/software/compiler/gcc/11.3.0/lib/gcc/x86_64-pc-linux-gnu/11.3.0/include-fixed /usr/include End of search list. cc1: error: CPU you selected does not support x86-64 instruction set GNU C17 (GCC) version 11.3.0 (x86_64-pc-linux-gnu) compiled by GNU C version 11.3.0, GMP version 6.1.0, MPFR version 3.1.6, MPC version 1.0.3, isl version isl-0.18-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 COLLECT_GCC_OPTIONS='-v' '-march=native' '-D' 'GEMM_MULTITHREAD_THRESHOLD=4' '-D' 'NO_PARALLEL_MAKE=0' '-o' 'getarch' '-dumpdir' 'getarch-' /workspace/software/compiler/gcc/11.3.0/libexec/gcc/x86_64-pc-linux-gnu/11.3.0/cc1 -E -lang-asm -quiet -v -D GEMM_MULTITHREAD_THRESHOLD=4 -D NO_PARALLEL_MAKE=0 cpuid.S -march=i386 -mno-mmx -mno-popcnt -mno-sse -mno-sse2 -mno-sse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -mno-avx -mno-avx2 -mno-sse4a -mno-fma4 -mno-xop -mno-fma -mno-avx512f -mno-bmi -mno-bmi2 -mno-aes -mno-pclmul -mno-avx512vl -mno-avx512bw -mno-avx512dq -mno-avx512cd -mno-avx512er -mno-avx512pf -mno-avx512vbmi -mno-avx512ifma -mno-avx5124vnniw -mno-avx5124fmaps -mno-avx512vpopcntdq -mno-avx512vbmi2 -mno-gfni -mno-vpclmulqdq -mno-avx512vnni -mno-avx512bitalg -mno-avx512bf16 -mno-avx512vp2intersect -mno-3dnow -mno-adx -mno-abm -mno-cldemote -mno-clflushopt -mno-clwb -mno-clzero -mno-cx16 -mno-enqcmd -mno-f16c -mno-fsgsbase -mno-fxsr -mno-hle -mno-sahf -mno-lwp -mno-lzcnt -mno-movbe -mno-movdir64b -mno-movdiri -mno-mwaitx -mno-pconfig -mno-pku -mno-prefetchwt1 -mno-prfchw -mno-ptwrite -mno-rdpid -mno-rdrnd -mno-rdseed -mno-rtm -mno-serialize -mno-sgx -mno-sha -mno-shstk -mno-tbm -mno-tsxldtrk -mno-vaes -mno-waitpkg -mno-wbnoinvd -mno-xsave -mno-xsavec -mno-xsaveopt -mno-xsaves -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -mno-uintr -mno-hreset -mno-kl -mno-widekl -mno-avxvnni -mtune=generic -fno-directives-only -o /tmp/ccVep3kP.s ignoring nonexistent directory "/workspace/software/compiler/gcc/11.3.0/lib/gcc/x86_64-pc-linux-gnu/11.3.0/../../../../x86_64-pc-linux-gnu/include"

include "..." search starts here:

include <...> search starts here:

/workspace/software/compiler/gcc/11.3.0/lib/gcc/x86_64-pc-linux-gnu/11.3.0/include /usr/local/include /workspace/software/compiler/gcc/11.3.0/include /workspace/software/compiler/gcc/11.3.0/lib/gcc/x86_64-pc-linux-gnu/11.3.0/include-fixed /usr/include End of search list. cc1: error: CPU you selected does not support x86-64 instruction set make: [Makefile.prebuild:70: getarch] Error 1 Makefile.system:1397: Makefile.: No such file or directory make: No rule to make target 'Makefile.'. Stop.

martin-frbg commented 7 months ago

I do not understand this - there is nothing in the Makefiles that would set -march=i386, this looks to be a default setting of your gcc - as if you built this gcc on a 32bit host (or maybe this is what happens when gcc or its build scripts cannot figure out the cpu type at all ?). Perhaps it will work if you specify something like CC="gcc -march=znver1"

yejianbang commented 7 months ago

I also find it strange that I didn't set any parameters for "-march" when I installed gcc11.But I don't know why the system recognizes it as i386.That surprises me.I also feel that I can't install openblas because of "-march=i386".

yejianbang commented 7 months ago

@martin-frbg I also don't understand why gcc11 compiled on "Hygon C86 7390 32-core Processor" machine automatically with "-march=i386". Is it something to do with his AMD chip? Maybe compile gcc to automatically recognize AMD chips as "i386"

I made the following changes in "Makefile.prebuild" and openblas successful compiled.

Add "-march=znver3" to the two lines that compile "getarch" and "getarch_2nd" to make it easy to recognize that I have a 64-bit x86 system

Finally, thank you again for your communication, I wish you a happy life!

martin-frbg commented 7 months ago

Good to know you got it working - as far as I can tell from the git log of the gcc source tree, Hygon cpu support may not have been added to the official GNU gcc sources. This would probably make gcc treat it as some early i486 processor when no -arch argument is supplied.

Rtoax commented 6 months ago

Good to know you got it working - as far as I can tell from the git log of the gcc source tree, Hygon cpu support may not have been added to the official GNU gcc sources. This would probably make gcc treat it as some early i486 processor when no -arch argument is supplied.

Thanks, I just got this too:

$ gcc -march=native -Q --help=target | grep -e '^  -march='
cc1: error: CPU you selected does not support x86-64 instruction set
  -march=                           i386
syaoo commented 3 months ago

I encountered the same problem on Hygon C86 5380 devices. I found that this is because the default value ’x86-64‘ of '-march' is changed to 'native' in the Makefile.system. After commenting GETARCH_LLAGS+=- march=native at Makefile.system, the compilation can be smoothly completed using 'make'.

befor commenting

# On x86_64 build getarch with march=native unless the compiler is PGI. This is required to detect AVX512 support in getarch.
ifeq ($(HOSTARCH), x86_64)
ifeq ($(findstring pgcc,$(HOSTCC))$(findstring nvc,$(HOSTCC)),)
GETARCH_FLAGS += -march=native
endif
endif

after commenting

# On x86_64 build getarch with march=native unless the compiler is PGI. This is required to detect AVX512 support in getarch.
ifeq ($(HOSTARCH), x86_64)
ifeq ($(findstring pgcc,$(HOSTCC))$(findstring nvc,$(HOSTCC)),)
#GETARCH_FLAGS += -march=native
endif
endif

here is compile getarch log use make CC="gcc -v" getarch >gcc-v_default.log 2>&1 befor commenting:

gcc-v_default.log

and this is compile getarch log after commenting:

gcc-v_comment.log

I don't understand why "-march=native" is not useful for the Hygon CPU, and this is a crude solution, but this is useful to me, I hope this information can also help those in need.

martin-frbg commented 3 months ago

Hygon detection by gcc appears to be completely broken in gcc12, in the past it used to report an ancient Xeon (so 64bit at least). Others have noticed this too - https://github.com/geoschem/GCHP/issues/391 Unfortunately I do not have access to any Hygon system to see if the problem is fixed in gcc13 or 14

syaoo commented 3 months ago

Hygon detection by gcc appears to be completely broken in gcc12, in the past it used to report an ancient Xeon (so 64bit at least). Others have noticed this too - geoschem/GCHP#391 Unfortunately I do not have access to any Hygon system to see if the problem is fixed in gcc13 or 14

I have tested Hygon C86 5380 that gcc 13.1.0 can compile normally.

here is complile getarch log: gcc-13.1.0.log