make OpenBLAS failed with USE_OPENMP=1 and USE_THREAD=0

[AIRicky]

Hi all, I am compiling the OpenBLAS on Ubuntu, however, I got the issue of "OpenBLAS: Cannot set both USE_OPENMP=1 and USE_THREAD=0. The USE_THREAD=0 is only for building single thread version.. Stop." and "recipe for target '/home/ricky/sdpa-c/OpenBLAS/libopenblas.a' failed". I am a beginner of OpenBLAS, could you help me figure this issue? Thank you so much. Here is the screen output:

================================================

ricky@ricky-virtual-machine:~/sdpa-c$ make OpenBLAS (cd /home/ricky/sdpa-c; \ rm -rf OpenBLAS; \ git clone git://github.com/xianyi/OpenBLAS; \ mv -f OpenBLAS /home/ricky/sdpa-c/OpenBLAS; \ cd /home/ricky/sdpa-c/OpenBLAS; \ make NO_WARMUP=1 USE_THREAD=1 USE_OPENMP=1 FC=gfortran CC=gcc libs netlib) Cloning into 'OpenBLAS'... remote: Counting objects: 32618, done. remote: Total 32618 (delta 0), reused 0 (delta 0), pack-reused 32618 Receiving objects: 100% (32618/32618), 21.39 MiB | 2.42 MiB/s, done. Resolving deltas: 100% (26112/26112), done. Checking connectivity... done. Checking out files: 100% (9021/9021), done. mv: cannot move 'OpenBLAS' to a subdirectory of itself, '/home/ricky/sdpa-c/OpenBLAS/OpenBLAS' make[1]: Entering directory '/home/ricky/sdpa-c/OpenBLAS' Makefile.system:412: OpenBLAS: Cannot set both USE_OPENMP=1 and USE_THREAD=0. The USE_THREAD=0 is only for building single thread version.. Stop. make[1]: Leaving directory '/home/ricky/sdpa-c/OpenBLAS' Makefile:83: recipe for target '/home/ricky/sdpa-c/OpenBLAS/libopenblas.a' failed make: [/home/ricky/sdpa-c/OpenBLAS/libopenblas.a] Error 2

[martin-frbg]

Seems you are using your own Makefile that fetches and compiles OpenBLAS ? I'd suspect there is some subtle mistake in that file, and the USE_THREAD=1 is not actually seen by "make".

[AIRicky]

Yeah, how can I fix this mistake in the make file? I found the command associated with OpenBLAS as below.

do not add '/' to the end of openblas_dir

openblas_dir = $(sdpac_dir)/OpenBLAS openblas_lib = $(openblas_dir)/libopenblas.a

sdpac_all_lib = $(suitesparse_lib) $(mumps_lib) $(openblas_lib)\ $(openmp_lib) $(fortran_lib) $(realtime_lib)

[martin-frbg]

Can you post the complete file ? The fragment you quoted has only some settings related to using OpenBLAS in your project, but somewhere there must be the line with make NO_WARMUP=1 etc.

[AIRicky]

Here is the complete file.

sdpac_dir = $(HOME)/sdpa-c

CC = gcc CXX = g++ FC = gfortran

CFLAGS = -O3 -fPIC CXXFLAGS = -O3 -fPIC -DOPENBLAS FCFLAGS = -O3 -fPIC

SDPAC_VERSION = \"7.3.8\" sdpac_lib = libsdpa-c.a sdpac_exe = sdpa-c

do not add '/' to the end of suitesparse_dir

suitesparse_dir = $(sdpac_dir)/SuiteSparse suitesparse_include = -I$(suitesparse_dir)/CHOLMOD/Include \ -I$(suitesparse_dir)/SuiteSparse_config suitesparse_lib = $(suitesparse_dir)/CHOLMOD/Lib/libcholmod.a \ $(suitesparse_dir)/CCOLAMD/Lib/libccolamd.a \ $(suitesparse_dir)/COLAMD/Lib/libcolamd.a \ $(suitesparse_dir)/CAMD/Lib/libcamd.a \ $(suitesparse_dir)/AMD/Lib/libamd.a \ $(suitesparse_dir)/metis-4.0/libmetis.a \ $(suitesparse_dir)/SuiteSparse_config/libsuitesparseconfig.a suitesparse_ver = 4.0.2

do not add '/' to the end of mumps_dir

mumps_dir = $(sdpac_dir)/mumps mumps_include = -I$(mumps_dir)/include mumps_lib = $(mumps_dir)/lib/libdmumps.a \ $(mumps_dir)/lib/libmumps_common.a \ $(mumps_dir)/lib/libpord.a \ $(mumps_dir)/libseq/libmpiseq.a mumps_ver = 4.10.0.dfsg mumps_tarfile = mumps${mumps_ver}.orig.tar.gz

do not add '/' to the end of openblas_dir

openblas_dir = $(sdpac_dir)/OpenBLAS openblas_lib = $(openblas_dir)/libopenblas.a

openmp_lib = -lgomp fortran_lib = -lgfortran realtime_lib = -lrt

sdpac_all_include = $(suitesparse_include) $(mumps_include) sdpac_all_lib = $(suitesparse_lib) $(mumps_lib) $(openblas_lib)\ $(openmp_lib) $(fortran_lib) $(realtime_lib)

[martin-frbg]

Hmm. Looks like this is not the Makefile itself, but a settings file included from it. Do you have a file named "Makefile" or "makefile" in /home/ricky/sdpa-c ?

[AIRicky]

==================================== -include make.inc -include make.obj .PHONY: SuiteSparse MUMPS OpenBLAS

all: $(sdpac_exe)

matlab: all (cd mex; make DEVELOPEMENT_STAGE=1) @echo === Add "$(sdpac_dir)/mex" to your Matlab path. ===

octave: all (cd mex; make DEVELOPEMENT_STAGE=1 COMPILE_ENVIRONMENT=octave) @echo === Add "$(sdpac_dir)/mex" to your octave path. ===

$(sdpac_exe): SuiteSparse MUMPS OpenBLAS $(sdpac_lib) sdpa_exe.o $(CXX) $(CXXFLAGS) -o $@ sdpa_exe.o $(sdpac_lib) $(sdpac_all_lib)

$(sdpac_lib): $(OBJ) rm -f $@ ar r $@ $(OBJ) ranlib $@

.cpp.o: $(CXX) $(CXXFLAGS) -c -o $@ -I. $(sdpac_all_include) \ -DVERSION=$(SDPAC_VERSION) $<

SuiteSparse:$(suitesparse_lib) $(suitesparse_lib): (cd $(sdpac_dir); \ rm -rf SuiteSparse; \ wget http://faculty.cse.tamu.edu/davis/SuiteSparse/SuiteSparse-$(suitesparse_ver).tar.gz; \ tar xzf SuiteSparse-$(suitesparse_ver).tar.gz; \ rm -f SuiteSparse-$(suitesparse_ver).tar.gz; \ mv -f SuiteSparse $(suitesparse_dir); \ cd $(suitesparse_dir); \ wget http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/OLD/metis-4.0.3.tar.gz; \ tar xzf metis-4.0.3.tar.gz; \ mv metis-4.0.3 metis-4.0; \ cd $(suitesparse_dir)/metis-4.0; \ echo CC = $(CC) >> Makefile.in; \ echo OPTFLAGS = $(CFLAGS) >> Makefile.in; \ make; \ cd $(suitesparse_dir); \ echo CC = $(CC) >> $(suitesparse_dir)/SuiteSparse_config/SuiteSparse_config.mk; \ echo CF = $(CFLAGS) -fexceptions >> $(suitesparse_dir)/SuiteSparse_config/SuiteSparse_config.mk; \ echo CC = $(CC) >> $(suitesparse_dir)/CSparse/Lib/Makefile; \ make)

MUMPS:$(mumps_lib) $(mumps_lib): (cd $(sdpac_dir); \ rm -rf mumps; \ wget wget http://ftp.de.debian.org/debian/pool/main/m/mumps/$(mumps_tar_file); \ tar xzf $(mumps_tar_file); \ rm -f $(mumps_tar_file); \ mv -f mumps-$(mumps_ver)/ $(mumps_dir); \ cd $(mumps_dir); \ cp Make.inc/Makefile.inc.generic.SEQ Makefile.inc; \ echo "CC = " $(CC) >> Makefile.inc; \ echo "FC = " $(FC) >> Makefile.inc; \ echo "FL = " $(FC) >> Makefile.inc; \ echo "OPTC = " $(CFLAGS) >> Makefile.inc; \ echo "OPTF = " $(FCFLAGS) >> Makefile.inc; \ echo "OPTL = " $(FCFLAGS) >> Makefile.inc; \ echo "LPORDDIR = "pwd"/PORD/lib/" >> Makefile.inc; \ echo "IPORDDIR = "pwd"/PORD/include/" >> Makefile.inc; \ make d;)

OpenBLAS:$(openblas_lib) $(openblas_lib): (cd $(sdpac_dir); \ rm -rf OpenBLAS; \ git clone git://github.com/xianyi/OpenBLAS; \ mv -f OpenBLAS $(openblas_dir); \ cd $(openblas_dir); \ make NO_WARMUP=1 USE_THREAD=1 USE_OPENMP=1 FC=$(FC) CC=$(CC) libs netlib)

clean: rm -rf ~ .o @echo "## If you want to clean SuiteSparse, MUMPS, OpenBLAS," @echo "## then use 'make dist-clean'"

dist-clean: clean rm -f $(sdpac_lib) $(sdpac_exe) (cd $sdpac_dir; \ rm -rf SuiteSparse; rm -rf mumps; rm -rf OpenBLAS) (cd mex; make dist-clean) -include make.head

[martin-frbg]

Strange. The make line there looks correct, no additional spaces or unusual characters in the USE_THREAD=1. The only place in the OpenBLAS build files where USE_THREAD is forcefully set to zero is (in OpenBLAS/Makefile.system) when it determines that the cpu has only a single core that is not capable of multithreading. (Now I notice above that the hostname is ricky-virtual-machine, that may explain it... and I think where the OpenBLAS makefile has "override USE_THREAD=0" it should also do "override USE_OPENMP=0" so that it does not trap itself a few lines later.) Perhaps the easiest solution for you - if you (or sdpa-c) want to have a multithreaded version of the library even if your current virtual machine cannot do multithreading - is to add "NUM_THREADS=4" in the sdpa-c Makefile where it has the NO_WARMUP=1 etc. Or change the configuration of your virtual machine so that it can run more than one thread.

[AIRicky]

It doesn't seem to work. Is it possible for you to try this package? Thank you so much.

sdpa-c.7.3.8.src.20130121.tar.gz

[martin-frbg]

Downloaded it, but it seems this will take a while to build. Had to fix the download source for SuiteSparse in the Makefile already.

[AIRicky]

Hmm, I think I need to check the Makefile carefully and I may re-write the Makefile to make it work. ^_^

[martin-frbg]

Makefile line 43 needs to be wget http://faculty.cse.tamu.edu/davis/SuiteSparse/SuiteSparse-$(suitesparse_ver).tar.gz; \ and in make.inc you need to add -lphtread at the end (or on the realtime_lib line after the -lrt). Regular build succeeded with these changes, now I need to trick it into finding only one core when it tries to build OpenBLAS so that I can replicate your problem.

[AIRicky]

It seems that the original problem still exists when I applied the two changes you offered. "OpenBLAS: Cannot set both USE_OPENMP=1 and USE_THREAD=0. The USE_THREAD=0 is only for building single thread version.. Stop." and "recipe for target '/home/ricky/sdpa-c/OpenBLAS/libopenblas.a' failed".

[martin-frbg]

Did you try adding NUM_THREADS=4 to the make options for OpenBLAS (on the "make NO_WARMUP=1..." line in the sdpa Makefile), or (temporarily) adding another cpu core to your virtual machine ? I have queued a patch for OpenBLAS to disable USE_OPENMP in OpenBLAS whenever USE_THREAD is automatically disabled. The other two changes are necessary to get that (five years old) sdpa package to compile at all in a current environment (suitesparse software moved to new download location, linkers no longer adding implied libraries by themselves)

[brada4]

Usually you get suitesparse and mumps with OpenBLAS or ATLAS from your linux distribution. In case of ubuntu you can tweak -lblas to actually use OpenBLAS.

[AIRicky]

It seems work. But another issue happened. I think the compile of OpenBLAS has finished.

g++ -O3 -fPIC -DOPENBLAS -o sdpa-c sdpa_exe.o libsdpa-c.a /home/ricky/sdpa-c/SuiteSparse/CHOLMOD/Lib/libcholmod.a /home/ricky/sdpa-c/SuiteSparse/CCOLAMD/Lib/libccolamd.a /home/ricky/sdpa-c/SuiteSparse/COLAMD/Lib/libcolamd.a /home/ricky/sdpa-c/SuiteSparse/CAMD/Lib/libcamd.a /home/ricky/sdpa-c/SuiteSparse/AMD/Lib/libamd.a /home/ricky/sdpa-c/SuiteSparse/metis-4.0/libmetis.a /home/ricky/sdpa-c/SuiteSparse/SuiteSparse_config/libsuitesparseconfig.a /home/ricky/sdpa-c/mumps/lib/libdmumps.a /home/ricky/sdpa-c/mumps/lib/libmumps_common.a /home/ricky/sdpa-c/mumps/lib/libpord.a /home/ricky/sdpa-c/mumps/libseq/libmpiseq.a /home/ricky/sdpa-c/OpenBLAS/libopenblas.a -lgomp -lgfortran -lrt /usr/bin/ld: libsdpa-c.a(sdpa_newton.o): undefined reference to symbol 'pthread_create@@GLIBC_2.2.5' //lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status Makefile:28: recipe for target 'sdpa-c' failed make: *** [sdpa-c] Error 1

[martin-frbg]

That last problem is why I suggested adding -lpthread after the -lrt in make.inc

[AIRicky]

Cool. All the problem have solved successfully. Thank you so much. ^_^