Open jianshu93 opened 1 year ago
put fn main() {println!("cargo:rustc-link-lib=framework=Accelerate") }
into a file build.rs
at the root of your directory and add features = ["accelerate"]
to the blas-src
package in your Cargo.toml
.
Hello @matthagan15 ,
I tried but it did not work because openblas is still linked. ndarray does rely on blas-src. So the problem is with openblas but not blas-src. This is how cargo.toml looks like:
ndarray = {version = "0.15", features=["rayon", "serde","blas"]} ndarray-linalg = {version = "0.16",default-features = false} blas-src = { version = "0.8", features = ["accelerate"] } openblas-src = { version = "0.10", features = ["cblas", "system"] }
[features]
default = []
intel-mkl-static = ["ndarray/blas","ndarray-linalg/intel-mkl-static"]
openblas-static = ["ndarray/blas", "ndarray-linalg/openblas-static"]
openblas-system = ["ndarray/blas", "ndarray-linalg/openblas-system"]
Thank,
Jianshu
With the above configuration, accelerate is actually working, but openblas is still linked, see my binary link status after using the openblas-system feature (via otool -L on macOS):
/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
/opt/homebrew/opt/openblas/lib/libopenblas.0.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
Thanks,
Jianshu
That is the openblas-system feature in ndarray-linalg is still using openblas-src system feature. Any idea why?
Thanks,
Jianshu
yes you should not be using openblas. use the feature "blas" on ndarray and use the crate "blas-src" with the feature "accelerate"
Hello @matthagan15 ,
But my question how to use accelerate backend to replace openblas, I am using openblas actually. That is to say accelerate is only for blas, but not for openblas, meaning I have no way to get rid of openblas right because I need both blas and openblas feature.
Thanks,
Jianshu
I also tried uninstall openblas on my system but has error with the same above configuration:
error: linking with cc
failed: exit status: 1
|
= note: LC_ALL="C" PATH="/Users/jianshuzhao/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/bin:/Users/jianshuzhao/miniconda3/condabin:/usr/local/bin:/Users/jianshuzhao/Github/bowtie2-2.5.0-macos-arm64:/opt/homebrew/bin:/Users/jianshuzhao/bin:/opt/homebrew/opt/bzip2/bin:/Users/jianshuzhao/Github/mummer-4.0.0beta5/bin:/Users/jianshuzhao/go/bin:/Users/jianshuzhao/Github/hmmer-h3-arm/bin:/opt/homebrew/opt/llvm/bin:/opt/homebrew/opt/openjdk/bin:/opt/homebrew/Cellar/coreutils/9.1/bin:/opt/homebrew/opt/ruby/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/Users/jianshuzhao/.cargo/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/usr/local/go/bin:/opt/X11/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin" VSLANG="1033" ZERO_AR_DATE="1" "cc" "-arch" "arm64" "/var/folders/mr/8qybn91j5fgdbrcdmq7ckftw0000gn/T/rustc3rP5ZO/symbols.o" "/Users/jianshuzhao/Documents/annembed/target/release/deps/embed-958d02413793165a.embed.3c7a159a26d98867-cgu.03.rcgu.o" "-L" "/Users/jianshuzhao/Documents/annembed/target/release/deps" "-L" "/opt/homebrew/opt/openblas/lib" "-L" "/opt/homebrew/opt/libomp/lib" "-L" "/Users/jianshuzhao/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "/Users/jianshuzhao/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-a2bfdcafe5010b58.rlib" "-framework" "Accelerate" "-lopenblas" "-liconv" "-lSystem" "-lc" "-lm" "-L" "/Users/jianshuzhao/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/Users/jianshuzhao/Documents/annembed/target/release/deps/embed-958d02413793165a" "-Wl,-dead_strip" "-nodefaultlibs"
= note: ld: warning: directory not found for option '-L/opt/homebrew/opt/openblas/lib'
ld: library not found for -lopenblas
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: could not compile annembed
(bin "embed") due to previous error
indicating that with this configuration, cc/clang is still try to find openblas instead of accelerate. Any idea.
Thanks,
Jianshu
Try this:
ndarray = {version = "0.15", features=["rayon", "serde","blas"]}
ndarray-linalg = {version = "0.16",default-features = false}
blas-src = { version = "0.8", features = ["accelerate"] }
[features]
default = []
and then add build.rs
with:
fn main() {println!("cargo:rustc-link-lib=framework=Accelerate")}
in it.
you should only be using one blas src.
Hello @matthagan15 ,
Many thanks! the problem is solved!! Greatly appreciated openblas must be uninstalled first and then it is successful:
/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0) /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
That is we do not specific any backend but let rust to choose and allow the accelerate in build.rs.
Thanks a again,
Jianshu
I'm pretty sure you don't have to uninstall openblas, I still have it installed via home brew, but if it worked for you then good!
Hello @matthagan15 ,
I have ran into another case where:
ndarray = {version = "0.15", features=["rayon", "serde","blas"]} ndarray-linalg = {version = "0.16", default-features = false} lapacke = {version = "0.5"}
lapack-src = {version = "0.8", features = ["accelerate"]} blas-src = {version = "0.8", features = ["accelerate"]}
I want to use the accelerate for lapack-src and blas-src, and as always ndarray and ndarray-linalg are all using the blas-src backend. However, I also need to use lapacke, I have the same build.rs mentioned above. I have the following error when linking (no errors if change back to openblas-src on the M1 Mac):
= note: Undefined symbols for architecture arm64: "_LAPACKE_dggsvd3", referenced from: graphembed::embed::atp::gsvd::GSvd$LT$F$GT$::do_gsvd::hd2a830ebb48b0ec3 in embed-11acb125d144fce5.embed.4061c9940487d9c0-cgu.09.rcgu.o ld: symbol(s) not found for architecture arm64 clang: error: linker command failed with exit code 1 (use -v to see invocation)
It seems of the the lapacke problem but not others. I tried both clang and gcc as the compiler since lapacke is just a rust wrapper around lapacke(C).
Many thanks,
Jianshu
Hello All, I want to bring this up again,
I recently ran into the ARM performance library (https://developer.arm.com/Tools%20and%20Software/Arm%20Performance%20Libraries) (or for MacOS: https://community.arm.com/arm-community-blogs/b/tools-software-ides-blog/posts/first-macos-release-of-arm-performance-libraries), which is just ARM version of OpenBLAS but is carefully designed only for ARM CPUs. MacOS is such an example but I can imagine there will be more Linux ARM CPUs in the future.
I open an issue here: https://github.com/rust-ndarray/ndarray-linalg/issues/378 to ask whether ndarray-linalg team will add support for it. It can be even faster as I can imagine. Just to let all you know that we can keep an eye on it, so that we can use it for both M-series MacOS and ARM Linux.
Thanks,
Jianshu
Dear ndarray-linalg team,
I am wondering whether the backend can be apple's accelerate framework. How can I setup and compile so that I can use the accelerate framework, which is well designed and optimized on apple devices.
Thanks,