Closed picn0113 closed 3 years ago
Could you supply build log? The line before libm error does not require it. record like with a command script
.
Please try adding a line with
EXTRALIB=-lm
in test/Makefile
There is no libSystem.dylib in /usr/lib.
ls /usr/lib
charset.alias libpython2.7.dylib
cron libstdc++.6.dylib
dsc_extractor.bundle libstdc++.dylib
dtrace log
dyld pam
groff php
libLeaksAtExit.dylib pkgconfig
libMTLCapture.dylib python2.7
libffi-trampolines.dylib rpcsvc
libgmalloc.dylib ruby
libhunspell-1.2.0.dylib sasl2
libhunspell-1.2.dylib sqlite3
libiodbc.2.dylib swift
libiodbc.dylib system
libiodbcinst.2.dylib updaters
libiodbcinst.dylib xpc
libobjc-trampolines.dylib zsh
libpython.dylib
I do not know much about the M1/OSX environment but these issues did not come up in earlier issue tickets, e.g. #2804 .
Me too. I think it's because ld failed to find the math library.
That should be possible to work around by adding the entry in test/Makefile and probably ctest/Makefile as well (though it still surprises me that this would come up only now)
That's because they haven't used gfortran. https://github.com/numpy/numpy/issues/17807#issuecomment-733402298
Hmm. Still should not preclude building OpenBLAS itself - unfortunately I do not have M1 hardware and as far as I know none of the usual CI services offers it yet. (The libSystem.dylib issue affects "only" NumPy, or does it stop the OpenBLAS build as well ?)
It's only for OpenBLAS. I even don't know why numpy needs OpenBLAS.
IIRC you could configure numpy to link to Apple's Accelerate or to the unoptimized "netlib" reference LAPACK and BLAS for matrix/vector functions. (Though with the reference implementation you would be likely to hit the same gfortran issues). Are you building on the M1 system itself, or cross-compiling for it on an Intel-based host ?
Just to clarify, do you run into the "missing libsystem.dylib" after fixing the Makefiles for the original libm problem, or where does this error come up ?
No cross-compiling, native build. It's just getting more errors. Sorry, I don't know where to add the line? There is no "missing libsystem.dylib", it's my guess.
test % gfortran -O2 -Wall -frecursive -fno-optimize-sibling-calls -march=armv8-a -o sblat1 sblat1.o ../libopenblas_armv8p-r0.3.12.dev.a -lm -L/usr/local/lib -lto_library -lSystem /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/lib/darwin/libclang_rt.osx.a
ld: library not found for -lm
collect2: error: ld returned 1 exit status
Around line 260 of test/Makefile you should have a line that starts with FLDFLAGS=
, you could put it after that line. And could you upload your Makefile.conf
that gets generated at the start of the build, please ?
possibly needs to end up behind the -L/usr/local/lib
, depending on where the libm is installed (or should it be a symlink to libsystem ?) As I said I do not know much about the platform, could also be you are missing (part of) some SDK ?
Of course there is no libm. I don't know the SDK. It's strange the system is lack of libraries.
test % make
gfortran -O2 -Wall -frecursive -fno-optimize-sibling-calls -fPIC -march=armv8-a -c sblat1.f -o sblat1.o
gfortran -O2 -Wall -frecursive -fno-optimize-sibling-calls -march=armv8-a -o sblat1 sblat1.o ../libopenblas_armv8p-r0.3.12.dev.a -lm -L/usr/local/lib -lto_library -lSystem /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/lib/darwin/libclang_rt.osx.a
ld: library not found for -lm
collect2: error: ld returned 1 exit status
make: *** [sblat1] Error 1
It is in /usr/lib/system, probably man abs
tells in which library original libm
is stuffed. libSystem (thats libc) , or some other.
Hm, strange. Somehow "it" thinks it is cross-compiling, but that should not break anything, only keep it from running the tests. And "it" already noticed the dependency on libm and put that in FEXTRALIB. No idea if libm should exist on OSX, but found https://github.com/phracker/MacOSX-SDKs which suggests that the SDK commandline tools (?) put it in /usr/lib as a symlink to a libSystem that seems to be a symlink to something else again
How to install the SDK? I have not installed the SDK.
/Library/Developer/CommandLineTools
This is where the CommandLineTools installed.
test % ls /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/lib
ACIPCBTLib.tbd libgmalloc.tbd
AppleConvergedTransport.tbd libgssapi_krb5.tbd
PN548.tbd libheimdal-asn1.tbd
PN548_API.tbd libhunspell-1.2.0.0.0.tbd
PN548_HAL.tbd libhunspell-1.2.0.tbd
bundle1.o libhunspell-1.2.tbd
crt1.10.5.o libiconv.2.4.0.tbd
crt1.10.6.o libiconv.2.tbd
crt1.o libiconv.tbd
dylib1.10.5.o libicucore.A.tbd
dylib1.o libicucore.tbd
lazydylib1.o libimg4.tbd
libAHTRestore.tbd libinfo.tbd
libATCommandStudioDynamic.tbd libiodbc.2.1.18.tbd
libAXSafeCategoryBundle.tbd libiodbc.2.tbd
libAccessibility.tbd libiodbc.tbd
libAccountPolicyTranslation.tbd libiodbcinst.2.1.18.tbd
libAppleArchive.tbd libiodbcinst.2.tbd
libAppleEXR.tbd libiodbcinst.tbd
libAppleSSEExt.tbd libipconfig.tbd
libAppletTranslationLibrary.tbd libipsec.A.tbd
libAudioIssueDetector.tbd libipsec.tbd
libAudioStatistics.tbd libk5crypto.tbd
libAudioToolboxUtility.tbd libkmod.a
libBSDPClient.A.tbd libkmodc++.a
libBSDPClient.tbd libkrb4.tbd
libCRFSuite.tbd libkrb5.tbd
libCRFSuite0.12.tbd libkrb524.tbd
libCTGreenTeaLogger.tbd libkrb5support.tbd
libCheckFix.tbd libktrace.tbd
libChineseTokenizer.tbd libl.a
libCoreStorage.tbd liblangid.tbd
libDHCPServer.A.tbd liblapack.tbd
libDHCPServer.tbd liblber.tbd
libDiagnosticMessagesClient.tbd libldap.tbd
libEndpointSecurity.tbd libldap_r.tbd
libFDR.tbd liblzma.5.tbd
libFDRDecode.tbd liblzma.tbd
libFDR_osx.tbd libm.tbd
libFaultOrdering.tbd libmarisa.tbd
libFosl_dynamic.tbd libmecab.tbd
libIASAuthReboot.tbd libmecabra.tbd
libIASUnifiedProgress.tbd libmenu.5.4.tbd
libIOAccessoryManager.tbd libmenu.tbd
libIOGTrace.tbd libmis.tbd
libIOReport.tbd libmorphun.tbd
libInFieldCollection.tbd libmx.A.tbd
libKernelCollectionBuilder.tbd libmx.tbd
libLeaksAtExit.tbd libncurses.5.4.tbd
libMTLCapture.tbd libncurses.5.tbd
libMatch.1.tbd libncurses.tbd
libMatch.tbd libnetsnmp.25.tbd
libMaxEnt.tbd libnetsnmp.tbd
libMemoryResourceException.tbd libnetsnmpagent.25.tbd
libMobileGestalt.tbd libnetsnmpagent.tbd
libParallelCompression.tbd libnetsnmphelpers.25.tbd
libQMIParserDynamic.tbd libnetsnmphelpers.tbd
libReverseProxyDevice.tbd libnetsnmpmibs.25.tbd
libSLAMDynamic.tbd libnetsnmpmibs.tbd
libSMC.tbd libnetsnmptrapd.25.tbd
libScreenReader.tbd libnetsnmptrapd.tbd
libSecRecord.tbd libnetwork.tbd
libSystem.B.tbd libnetworkextension.tbd
libSystem.B_asan.tbd libnfrestore.tbd
libSystem.tbd libnfshared.tbd
libSystem_asan.tbd libnfstorage.tbd
libTelephonyUtilDynamic.tbd libobjc-trampolines.tbd
libThaiTokenizer.tbd libobjc.A.tbd
libThreadExternalCommissioner.tbd libobjc.tbd
libUniversalAccess.tbd libodfde.tbd
libUpdateMetrics.a libodmodule.tbd
libXSEvent.tbd libpam.1.tbd
libXplugin.1.tbd libpam.2.tbd
libXplugin.tbd libpam.tbd
libamsupport.tbd libpanel.5.4.tbd
libapp_launch_measurement.tbd libpanel.tbd
libapple_nghttp2.tbd libpartition2_dynamic.tbd
libapr-1.0.tbd libpcap.A.tbd
libapr-1.tbd libpcap.tbd
libaprutil-1.0.tbd libpcre.0.tbd
libaprutil-1.tbd libpcre.tbd
libarchive.2.tbd libpcreposix.0.tbd
libarchive.tbd libpcreposix.tbd
libate.tbd libperfcheck.tbd
libauditd.0.tbd libpkstart.a
libauditd.tbd libpmenergy.tbd
libauthinstall.tbd libpmsample.tbd
libblas.tbd libpoll.tbd
libbootpolicy.tbd libprequelite.tbd
libboringssl.tbd libproc.tbd
libbsm.0.tbd libpthread.tbd
libbsm.tbd libpython.tbd
libbz2.1.0.5.tbd libpython2.7.tbd
libbz2.1.0.tbd libquic-asan-static.a
libbz2.tbd libquic.tbd
libc++.1.tbd libquit.tbd
libc++.tbd libreadline.tbd
libc++abi.tbd libresolv.9.tbd
libc.tbd libresolv.tbd
libcblas.tbd librpcsvc.tbd
libcharset.1.0.0.tbd libruby.2.6.tbd
libcharset.1.tbd libruby.tbd
libcharset.tbd libsandbox.1.tbd
libclapack.tbd libsandbox.tbd
libcmph.tbd libsasl2.2.0.1.tbd
libcodedirectory.tbd libsasl2.2.0.15.tbd
libcodedirectory_static.a libsasl2.2.0.21.tbd
libcom_err.tbd libsasl2.2.0.22.tbd
libcompression.tbd libsasl2.2.tbd
libcoreroutine.tbd libsasl2.tbd
libcoretls.tbd libspindump.tbd
libcoretls_cfhelpers.tbd libsqlite3.0.tbd
libcryptex.tbd libsqlite3.tbd
libcryptex_core.tbd libsysdiagnose.tbd
libcryptex_interface.tbd libsysmon.tbd
libcsfde.tbd libsystemstats.tbd
libcupolicy.tbd libtailspin.tbd
libcups.2.tbd libtcl.tbd
libcups.tbd libtcl8.5.tbd
libcupsimage.2.tbd libtclstub8.5.a
libcupsimage.tbd libtermcap.tbd
libcurl.3.tbd libtidy.A.tbd
libcurl.4.tbd libtidy.tbd
libcurl.tbd libtk.tbd
libcurses.tbd libtk8.5.tbd
libdbm.tbd libtkstub8.5.a
libdes425.tbd libtzupdate.tbd
libdl.tbd libusrtcp.tbd
libdns_services.tbd libutil.tbd
libdpfu.tbd libutil1.0.tbd
libdscsym.tbd libxar.1.tbd
libdtrace.tbd libxar.tbd
libdz.tbd libxcselect.tbd
libeasyperf.tbd libxml2.2.tbd
libedit.2.tbd libxml2.tbd
libedit.3.0.tbd libxpc_datastores.tbd
libedit.3.tbd libxslt.1.tbd
libedit.tbd libxslt.tbd
libenergytrace.tbd liby.a
libexpat.1.tbd libz.1.1.3.tbd
libexpat.tbd libz.1.2.11.tbd
libexslt.0.tbd libz.1.2.5.tbd
libexslt.tbd libz.1.2.8.tbd
libextension.tbd libz.1.tbd
libf77lapack.tbd libz.tbd
libfakelink.tbd log
libffi-trampolines.tbd php
libffi.tbd ssh-keychain.tbd
libform.5.4.tbd swift
libform.tbd system
libgcc_s.1.tbd updaters
libgermantok.tbd
No idea why gfortran fails to find them then - the gfortran is the one that @Keno linked to in #2814 I guess ?
Your link seems to be a build based on the one I mentioned, but if you look at unresolved issue 12 there https://github.com/fxcoudert/gfortran-for-macOS/issues/12 it appears to describe the same problem.
What do you get when compiling a simple hello world fortran program with gfortran -v hello.f90
?
gfortran -v hello.f90
Driving: gfortran -v hello.f90 -mmacosx-version-min=11.0.0 -asm_macosx_version_min=11.0 -l gfortran
Using built-in specs.
COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=/usr/local/gfortran/libexec/gcc/aarch64-apple-darwin20/11.0.0/lto-wrapper
Target: aarch64-apple-darwin20
Configured with: ../../gcc-darwin-arm64/configure --prefix=/usr/local/gfortran --with-gmp=/Users/fx/build/deps --with-isl=/Users/fx/build/deps --enable-languages=c,c++,fortran,objc,obj-c++ --build=aarch64-apple-darwin20 --with-native-system-header-dir=/usr/include --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 11.0.0 20201114 (experimental) (GCC)
COLLECT_GCC_OPTIONS='-v' '-mmacosx-version-min=11.0.0' '-asm_macosx_version_min=11.0' '-mlittle-endian' '-mabi=lp64' '-dumpdir' 'a-'
/usr/local/gfortran/libexec/gcc/aarch64-apple-darwin20/11.0.0/f951 hello.f90 -fPIC -quiet -dumpdir a- -dumpbase hello.f90 -dumpbase-ext .f90 -mmacosx-version-min=11.0.0 -mlittle-endian -mabi=lp64 -version -fintrinsic-modules-path /usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/11.0.0/finclude -o /var/folders/qx/xnxcrrlj4p52w9zy976f5prw0000gn/T//ccCXdfPR.s
GNU Fortran (GCC) version 11.0.0 20201114 (experimental) (aarch64-apple-darwin20)
compiled by GNU C version 11.0.0 20201114 (experimental), GMP version 6.2.0, MPFR version 4.1.0, MPC version 1.2.1, isl version isl-0.22.1-GMP
GGC heuristics: --param ggc-min-expand=30 --param ggc-min-heapsize=4096
GNU Fortran2008 (GCC) version 11.0.0 20201114 (experimental) (aarch64-apple-darwin20)
compiled by GNU C version 11.0.0 20201114 (experimental), GMP version 6.2.0, MPFR version 4.1.0, MPC version 1.2.1, isl version isl-0.22.1-GMP
GGC heuristics: --param ggc-min-expand=30 --param ggc-min-heapsize=4096
COLLECT_GCC_OPTIONS='-v' '-mmacosx-version-min=11.0.0' '-mlittle-endian' '-mabi=lp64' '-dumpdir' 'a-'
as -arch arm64 -v -mmacosx-version-min=11.0 -o /var/folders/qx/xnxcrrlj4p52w9zy976f5prw0000gn/T//ccpDa1oc.o /var/folders/qx/xnxcrrlj4p52w9zy976f5prw0000gn/T//ccCXdfPR.s
Apple clang version 12.0.0 (clang-1200.0.32.27)
Target: aarch64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
"/Library/Developer/CommandLineTools/usr/bin/clang" -cc1as -triple arm64-apple-macosx11.0.0 -filetype obj -main-file-name ccCXdfPR.s -target-cpu vortex -target-feature +v8.3a -target-feature +fp-armv8 -target-feature +neon -target-feature +crc -target-feature +crypto -target-feature +fullfp16 -target-feature +ras -target-feature +lse -target-feature +rdm -target-feature +rcpc -target-feature +zcm -target-feature +zcz -target-feature +sha2 -target-feature +aes -fdebug-compilation-dir /tmp -dwarf-debug-producer "Apple clang version 12.0.0 (clang-1200.0.32.27)" -dwarf-version=4 -mrelocation-model pic -o /var/folders/qx/xnxcrrlj4p52w9zy976f5prw0000gn/T//ccpDa1oc.o /var/folders/qx/xnxcrrlj4p52w9zy976f5prw0000gn/T//ccCXdfPR.s
Reading specs from /usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/11.0.0/../../../libgfortran.spec
rename spec lib to liborig
COLLECT_GCC_OPTIONS='-v' '-mmacosx-version-min=11.0.0' '-mlittle-endian' '-mabi=lp64' '-dumpdir' 'a-'
COMPILER_PATH=/usr/local/gfortran/libexec/gcc/aarch64-apple-darwin20/11.0.0/:/usr/local/gfortran/libexec/gcc/aarch64-apple-darwin20/11.0.0/:/usr/local/gfortran/libexec/gcc/aarch64-apple-darwin20/:/usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/11.0.0/:/usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/
LIBRARY_PATH=/usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/11.0.0/:/usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/11.0.0/../../../
COLLECT_GCC_OPTIONS='-v' '-mmacosx-version-min=11.0.0' '-mlittle-endian' '-mabi=lp64' '-dumpdir' 'a.'
/usr/local/gfortran/libexec/gcc/aarch64-apple-darwin20/11.0.0/collect2 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk/ -dynamic -arch arm64 -macosx_version_min 11.0.0 -weak_reference_mismatches non-weak -o a.out -L/usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/11.0.0 -L/usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/11.0.0/../../.. /var/folders/qx/xnxcrrlj4p52w9zy976f5prw0000gn/T//ccpDa1oc.o -lgfortran -lemutls_w -lgcc -lm -lemutls_w -lgcc -lSystem -lgcc -no_compact_unwind -v
collect2 version 11.0.0 20201114 (experimental)
/usr/bin/ld -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk/ -dynamic -arch arm64 -macosx_version_min 11.0.0 -weak_reference_mismatches non-weak -o a.out -L/usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/11.0.0 -L/usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/11.0.0/../../.. /var/folders/qx/xnxcrrlj4p52w9zy976f5prw0000gn/T//ccpDa1oc.o -lgfortran -lemutls_w -lgcc -lm -lemutls_w -lgcc -lSystem -lgcc -no_compact_unwind -v
@(#)PROGRAM:ld PROJECT:ld64-609.7
BUILD 18:10:15 Oct 19 2020
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
Library search paths:
/usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/11.0.0
/usr/local/gfortran/lib
Framework search paths:
ld: library not found for -lm
collect2: error: ld returned 1 exit status
It looks like gfortran was compiled with /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk
and gfortran saves this location into its binary. You have /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk
which becomes a problem.
A workaround is to do export SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk
.
This is not an issue with openblas, but rather gfortran.
ld: warning: object file (/usr/local/gfortran/lib/gcc/aarch64-apple-darwin20/11.0.0/libgcc.a(enable-execute-stack.o)) was built for newer macOS version (11.0) than being linked (10.8)
Undefined symbols for architecture arm64:
"___chkstk_darwin", referenced from:
_sgemv_ in libopenblas_armv8p-r0.3.12.dev.a(sgemv.o)
_sger_ in libopenblas_armv8p-r0.3.12.dev.a(sger.o)
_cblas_sgemv in libopenblas_armv8p-r0.3.12.dev.a(cblas_sgemv.o)
_cblas_sger in libopenblas_armv8p-r0.3.12.dev.a(cblas_sger.o)
_dgemv_ in libopenblas_armv8p-r0.3.12.dev.a(dgemv.o)
_dger_ in libopenblas_armv8p-r0.3.12.dev.a(dger.o)
_cblas_dgemv in libopenblas_armv8p-r0.3.12.dev.a(cblas_dgemv.o)
...
ld: symbol(s) not found for architecture arm64
collect2: error: ld returned 1 exit status
make[1]: *** [libopenblas_armv8p-r0.3.12.dev.dylib] Error 1
make: *** [shared] Error 2
export MACOSX_DEPLOYMENT_TARGET=11.0
OpenBLAS build complete. (BLAS CBLAS LAPACK LAPACKE)
OS ... Darwin
Architecture ... arm64
BINARY ... 64bit
C compiler ... CLANG (cmd & version : Apple clang version 12.0.0 (clang-1200.0.32.27))
Fortran compiler ... GFORTRAN (cmd & version : GNU Fortran (GCC) 11.0.0 20201114 (experimental))
-n Library Name ... libopenblas_armv8p-r0.3.12.dev.a
(Multi-threading; Max num-threads is 8)
WARNING: If you plan to use the dynamic library libopenblas_armv8p-r0.3.12.dev.dylib, you must run:
"make PREFIX=/your_installation_path/ install".
(or set PREFIX in Makefile.rule and run make install. If you want to move the .dylib to a new location later, make sure you change the internal name of the dylib with:
install_name_tool -id /new/absolute/path/to/libopenblas_armv8p-r0.3.12.dev.dylib libopenblas_armv8p-r0.3.12.dev.dylib
To install the library, you can run "make PREFIX=/path/to/your/installation install".
Numpy needs OpenBLAS for building.
make