autumnai / collenchyma

Extendable HPC-Framework for CUDA, OpenCL and common CPU
http://autumnai.github.io/collenchyma
Apache License 2.0
475 stars 33 forks source link

Support OS X. #46

Open y-ich opened 8 years ago

y-ich commented 8 years ago

Hi.

I did "cargo test" on my Mac and failed by the following error.

ld: library not found for -lOpenCL

Changing the code in src/frameworks/opencl/ffi.rs from

#[link(name = "OpenCL")]

to

#[cfg(target_os = "macos")]
#[link(name = "OpenCL", kind = "framework")]

solved the problem though I have no idea to support both linux and macos.

Next I got the following error

ld: library not found for -lcuda

I had installed cuda via HomeBrew. So it seems I need to pass library path to the linker. I tried link_args attribute in src/frameworks/cuda/api/ffi.rs, and rustc-link-search in Cargo.toml, but failed.

I will be happy if the library supports OS X.

hobofan commented 8 years ago

For OpenCl an itermediate solution should be:

#[cfg(not(target_os = "macos"))]
#[link(name = "OpenCL")]
#[cfg(target_os = "macos")]
#[link(name = "OpenCL", kind = "framework")]

I am not sure if we can support the required linker arguments with the current structure of the crate. Ideally with https://github.com/autumnai/collenchyma/issues/4 and https://github.com/autumnai/collenchyma/issues/3 and their accompanying -sys crates supplying linker args would become much easier.

Before resolving those issues we could also provide a build.rs directly in collenchyma similar to this one, but as I said I am not sure if it works with the current structure.

y-ich commented 8 years ago

Thank you!

Now I solved these only for mac by build.rs and could compiled both collenchyma and leaf.

I am looking forward to your Mac support^^

hobofan commented 8 years ago

Did you have to make any significant changes to the build.rs? Which command (with ENV variables) did you have to run to get it to compile?

I think a short gist showing it would be nice for others to get it to run and as a base for a future PR.

y-ich commented 8 years ago

It may be difficult to make generic form because library path depends its environment and I wrote "hard-wired" code.

I installed cuda via HomeBrew and wrote build.rs for collenchyma as below.

fn main() {
    println!("cargo:rustc-link-search=native=/usr/local/cuda/lib");
}

And I succeeded to compile but failed framework_cuda_spec::it_allocates_4gb_memory_same_device test.

And for leaf, I wrote build.rs as below,

fn main() {
    println!("cargo:rustc-link-lib=framework=OpenCL");
}

I am not quite sure why I need it here. It seems #[link(name = "OpenCL", kind = "framework")] in collenchyma may not work when a package (leaf) use it. All tests for leaf passed but I am not sure if cuda and opencl work well since you may implement fall back.

This is all information that I have now.

hobofan commented 8 years ago

Might have been resolved by #52. Feedback if it's working without a build.rs is appreciated!

reem commented 8 years ago

This is not working for me without a build.rs, I receive a build failure when ld cannot find -lcuda.

Note that when we do get this working it should be possible to test it on travis by adding osx as an os for testing.

reem commented 8 years ago

@y-ich hmm, I tried your proposed workaround and I still cannot get a clean compile, here's my attempted build.rs and the resulting failure:

build.rs:

fn main() {
    println!("cargo:rustc-link-search=native=/usr/local/cuda/lib");
}

failure:

error: linking with `cc` failed: exit code: 1
note: "cc" "-m64" "-L" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib" "/Users/reem/autumn/collenchyma/target/debug/shared_memory_specs-0a57c95f58ada7a2.0.o" "-o" "/Users/reem/autumn/collenchyma/target/debug/shared_memory_specs-0a57c95f58ada7a2" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/Users/reem/autumn/collenchyma/target/debug" "-L" "/Users/reem/autumn/collenchyma/target/debug/deps" "-L" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib/libtest-18402db3.rlib" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib/libterm-18402db3.rlib" "/Users/reem/autumn/collenchyma/target/debug/libcollenchyma.rlib" "/Users/reem/autumn/collenchyma/target/debug/deps/libenum_primitive-7365ecacb96a7c97.rlib" "/Users/reem/autumn/collenchyma/target/debug/deps/libnum-a2e6e61627ca7fe5.rlib" "/Users/reem/autumn/collenchyma/target/debug/deps/libbitflags-d06009685951cc56.rlib" "/Users/reem/autumn/collenchyma/target/debug/deps/librand-c724acb3942597d1.rlib" "/Users/reem/autumn/collenchyma/target/debug/deps/libbyteorder-79579e11a5fc0173.rlib" "/Users/reem/autumn/collenchyma/target/debug/deps/liblibc-036fbedefddee9e8.rlib" "/Users/reem/autumn/collenchyma/target/debug/deps/librustc_serialize-e1b49f9d5f55eb83.rlib" "/Users/reem/autumn/collenchyma/target/debug/deps/liblinear_map-c2c2927b059a7fd8.rlib" "/Users/reem/autumn/collenchyma/target/debug/deps/liblazy_static-553d4fa9dca2851d.rlib" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib/libgetopts-18402db3.rlib" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib/libstd-18402db3.rlib" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib/libcollections-18402db3.rlib" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib/librustc_unicode-18402db3.rlib" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib/librand-18402db3.rlib" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib/liballoc-18402db3.rlib" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-18402db3.rlib" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib/liblibc-18402db3.rlib" "/Users/reem/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib/libcore-18402db3.rlib" "-framework" "OpenCL" "-l" "cuda" "-l" "System" "-l" "pthread" "-l" "c" "-l" "m" "-Wl,-rpath,@loader_path/../../../../.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib" "-Wl,-rpath,/usr/local/lib/rustlib/x86_64-apple-darwin/lib" "-l" "compiler-rt"
note: ld: library not found for -lcuda
clang: error: linker command failed with exit code 1 (use -v to see invocation)

error: aborting due to previous error

I have cuda installed via homebrew at /usr/local/cuda.

MichaelHirn commented 8 years ago

So @hobofan and I just bounced some ideas on how to solve the build errors that keep popping up due to the third-party libraries (mostly CUDA, though).

There are two problems that we need to solve: the linking problem at build time and the linking problem at run time.

Linking at build time

One of the biggest problems is, which also causes most of the issues we receive, that different platforms have different locations for their CUDA library. This causes a lot of confusion and makes the issues hard to debug as it requires to be on that platform. (e.g. we can't really recreate OSX issues).

Few days ago we spoke to NVIDIA, and learned, that their libraries are available via a direct download link, which can help to solve the linking at build time situation. Our proposed plan forward is,

creating a X-sys crate, which takes care of downloading/linking to the expected X library. This should work on any platform with no to very little platform adjustment.

Now, that we know where the libraries are located, the build time linking is working reliable on any platform.

Linking at run time

As we would download the libraries in the build process to an artifical directory, there is definitly no way that ld would find the libraries in the ld-config path for run time linking - when we want to run the executable.

Two solutions for the run time linking

Feedback to the proposed solution is highly welcome. So far, this is our current take on how we would solve the build error issues.

@reem I did not know that this was possible with Travis - I like the idea. And with the download links from NVIDIA, (I think) this could be tested in a 'real' environment.

reem commented 8 years ago

One thing to keep in mind about possibly testing on travis is I'm not sure if the nodes actually have access to a GPU (I doubt it).

That sounds like a pretty good plan, I would suggest having a check for an already installed system library before downloading every time (downloading every time can be very slow), and also statically linking as much as possible to simplify deployment.

Anything I can do here to help get this working? I spun up a debian instance last night on GCE to get a linux dev environment going only to discover that GCE doesn't support GPU nodes...

MichaelHirn commented 8 years ago

Ohhh, this is unfortunate :) AWS has GPU nodes and Azure should have them since Dec, too. (although the comments say, it didn't launch yet).

But first, thanks for the PR to fix it on OSX - very much appreciated. I like the .travis.yml file but would try to already implement the basic concepts of the proposed build time approach, before merging it into master. (Skipping, for now, the X-sys crate part). Including the checking for already downloaded files on the machine and the download fallback in cases where the files couldn't be found.

I will write some more details regarding the PR there at #59

reem commented 8 years ago

Building OSX with --features native, but not opencl or cuda, is now tested on travis

MichaelHirn commented 8 years ago

Well, homu rushed a little bit too much here.

@reem opened #60, which tracks the CI support matrix for platforms (osx, linux) and frameworks (native, opencl, cuda).

I would leave this open until we have osx fully supported.

phdoerfler commented 8 years ago

Please excuse me if this is a dumb question but is the OSX support in collenchyma = "0.0.8" already? I tried getting the example in the README to work but it (still) fails with note: ld: library not found for -lOpenCL.