search

dask / dask-searchcv

dask-searchcv is now part of dask-ml: https://github.com/dask/dask-ml
BSD 3-Clause "New" or "Revised" License
240 stars 43 forks source link

The horrible journey to building Elemental with conda on OS X #6

Closed ahmadia closed 7 years ago

ahmadia commented 8 years ago

should be significantly easier for Linux, but maybe not.

Problem #1, cmake doesn't like conda's MPI very much, so we'll have to hardcode some MPI variables into the build system. Here's what it looks like on my box:

cmake -D CMAKE_INSTALL_PREFIX="~/anaconda3/envs/elemental" \
            -D MPI_C_INCLUDE_PATH:STRING=/Users/qmj240/anaconda3/envs/elemental/include \
            -D MPI_C_LINK_FLAGS:STRING=-L/Users/qmj240/anaconda3/envs/elemental/lib/ \
            -D "MPI_C_LIBRARIES:STRING=-lmpi -lpmpi \
            -L/Users/qmj240/anaconda3/envs/elemental/lib/" ..
ahmadia commented 8 years ago

changes needed in Elemental are raised as an issue here: https://github.com/elemental/Elemental/issues/149

ahmadia commented 8 years ago

make -j 4 seems to be safe.

ahmadia commented 8 years ago

This is a C1 proxy problem.

qmj240@f45c899d5ff9 ~/Elemental/build master $ cd source/
qmj240@f45c899d5ff9 ~/Elemental/build/source master $ git clone https://github.com/scibuilder/parmetis.git
ahmadia commented 8 years ago

Also much needed:

-D CMAKE_INSTALL_PREFIX="~/anaconda3/envs/elemental"
ahmadia commented 8 years ago

Still to do, get a proper conda build recipe for elemental so that we can just build this correctly once and never again. Currently stuck at Python 2 but shouldn't be hard to fix to Python 3 compatibility.

jcrist commented 7 years ago

The scope of this project has been reduced to just model parallelism (e.g. grid-search) over small-medium data. It'd still be nice to get this working ,but I'm closing for now, as this is unrelated to this project.

mrocklin commented 7 years ago

@ahmadia if you still have an interest in this I've been building a variety of dask-other projects recently where other is a distributed system. I'd be happy to collaborate here if handing off data between systems is still of value to you all and you have someone on the elemental side that is interested in taking this on.