search

pratikvn / schwarz-lib

Repository for testing asynchronous schwarz methods.
https://pratikvn.github.io/schwarz-lib/
BSD 3-Clause "New" or "Revised" License
5 stars 3 forks source link

Installation #2

Closed soumyadipghosh closed 5 years ago

soumyadipghosh commented 5 years ago

@pratikvn Few queries about installation:

1) How do I get gingko ? Do I just clone the gingko repo? Also in which file do I set the Gingko_DIR variable ? 2) For MPI, I usually do a module load $mpi version$ in our cluster. Will that work or do I need to set a path somewhere? 3) For Boost, similar question as above ? 4) For Metis, similar question as above ?

pratikvn commented 5 years ago

  1. You can get ginkgo by just cloning it normally and then building and installing it to some path you want as given in the Ginkgo installation page. Once you have installed the ginkgo in say foo_dir, you just set Ginkgo_DIR in the environment variable using export Ginkgo_DIR=/path/to/foo_dir.

  2. module load $mpi_version$ should be enough. Cmake's find_package(MPI) defined in the main CMakeLists.txt file should automatically find the MPI headers and the library and export it into the required variables.

  3. Same for Boost, if it is available as a module, you can load it and that should work.

  4. METIS is also the same but you would need to do export METIS_DIR= /path/to/installation

pratikvn commented 5 years ago

Additionally, if you are pushing something, please create a branch from develop (something like event-based) and push to that branch. Please dont push to develop.

soumyadipghosh commented 5 years ago

Building Gingko is failing. I am not sure how to send you the error messages, does it write a log file somewhere ? I did import mpi and boost before running. Additionally I used the basic build command specified here without specifying any options.

Also, I created a branch called event-based and push to it in future.

pratikvn commented 5 years ago

In the build folder, there should be a CMakeCache.txt file which holds the settings and the variables that cmake sets. You can see there if any of the variables are incorrectly set. Additionally, more detailed logs are available in build/CMakeFiles/CMakeError.log and build/CMakeFiles/CMakeOutput.log.

You can drag and drop the files when you comment here. That should automatically upload the file.

soumyadipghosh commented 5 years ago

Uploading the files here CMakeCache.txt CMakeError.log CMakeOutput.log

pratikvn commented 5 years ago

What errors do you get on the command line ? You can add collapsible stuff inside a block in markdown like this:

Click to expand ``` Here you add the output from your command line ```

You can see how the above collapsible block was written by clicking the three dots on the top right of this comment and trying to edit my comment. Then you see it in raw text. Selecting the preview tab shows you the processed text.

soumyadipghosh commented 5 years ago

cmake -G "Unix Makefiles" ..

Output ``` -- Could NOT find PAPI (missing: PAPI_LIBRARY PAPI_INCLUDE_DIR) -- Configuring done -- Generating done -- Build files have been written to: /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/build/third_party/gtest/download [ 11%] Skipping update step for 'gtest_external' [ 22%] Performing configure step for 'gtest_external' -- Configuring done -- Generating done -- Build files have been written to: /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/build/third_party/gtest/build [ 33%] Performing build step for 'gtest_external' [ 25%] Built target gtest [ 50%] Built target gmock [ 75%] Built target gmock_main [100%] Built target gtest_main [ 44%] No install step for 'gtest_external' [ 55%] No test step for 'gtest_external' [ 66%] Completed 'gtest_external' [100%] Built target gtest_external -- Configuring done -- Generating done -- Build files have been written to: /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/build/third_party/git-cmake-format/download [ 11%] Skipping update step for 'git-cmake-format' [ 22%] Performing configure step for 'git-cmake-format' -- Configuring done -- Generating done -- Build files have been written to: /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/build/third_party/git-cmake-format/build [ 33%] Performing build step for 'git-cmake-format' [ 44%] No install step for 'git-cmake-format' [ 55%] No test step for 'git-cmake-format' [ 66%] Completed 'git-cmake-format' [100%] Built target git-cmake-format -- Configuring done -- Generating done -- Build files have been written to: /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/build/third_party/gflags/download [ 11%] Skipping update step for 'gflags_external' [ 22%] Performing configure step for 'gflags_external' -- Configuring done -- Generating done -- Build files have been written to: /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/build/third_party/gflags/build [ 33%] Performing build step for 'gflags_external' [100%] Built target gflags_nothreads_static [ 44%] No install step for 'gflags_external' [ 55%] No test step for 'gflags_external' [ 66%] Completed 'gflags_external' [100%] Built target gflags_external -- Configuring done -- Generating done -- Build files have been written to: /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/build/third_party/rapidjson/download [ 11%] Skipping update step for 'rapidjson_external' [ 22%] Performing configure step for 'rapidjson_external' -- Configuring done -- Generating done -- Build files have been written to: /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/build/third_party/rapidjson/build [ 33%] Performing build step for 'rapidjson_external' [ 44%] No install step for 'rapidjson_external' [ 55%] No test step for 'rapidjson_external' [ 66%] Completed 'rapidjson_external' [100%] Built target rapidjson_external -- Configuring done -- Generating done -- Build files have been written to: /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/build ```

make

Output ``` [ 0%] Built target generate_ginkgo_header [ 0%] Built target ginkgo_reference_device [ 0%] Building CXX object reference/CMakeFiles/ginkgo_reference.dir/base/version.cpp.o In file included from /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/reference/base/version.cpp:33:0: /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp: In constructor ‘gko::version_info::version_info()’: /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: cannot convert ‘gko::version’ to ‘const uint64 {aka const long unsigned int}’ in initialization cuda_version{get_cuda_version()} ^ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::minor’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::patch’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::tag’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: cannot convert ‘gko::version’ to ‘const uint64 {aka const long unsigned int}’ in initialization /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::minor’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::patch’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::tag’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: cannot convert ‘gko::version’ to ‘const uint64 {aka const long unsigned int}’ in initialization /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::minor’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::patch’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::tag’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: cannot convert ‘gko::version’ to ‘const uint64 {aka const long unsigned int}’ in initialization /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::minor’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::patch’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::tag’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: cannot convert ‘gko::version’ to ‘const uint64 {aka const long unsigned int}’ in initialization /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::minor’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::patch’ /afs/crc.nd.edu/user/s/sghosh2/Public/ginkgo/include/ginkgo/core/base/version.hpp:207:42: error: uninitialized const member ‘gko::version::tag’ make[2]: *** [reference/CMakeFiles/ginkgo_reference.dir/base/version.cpp.o] Error 1 make[1]: *** [reference/CMakeFiles/ginkgo_reference.dir/all] Error 2 make: *** [all] Error 2 ```

The first command works fine I guess but errors show up in the second. Before executing these commands, I did git checkout master to select the master branch

pratikvn commented 5 years ago

Did you do the git checkout master after the cmake command ?

soumyadipghosh commented 5 years ago

No, before cmake.

pratikvn commented 5 years ago

Hm interesting, In any case, can you rm -rf the build directory and do the build process again ?

soumyadipghosh commented 5 years ago

I did it, same thing again! What is the minimum GNU compiler requirement? I am using GNU 4.8.5

pratikvn commented 5 years ago

Ah, that wont work. You need atleast a proper C++11 compliant one. Do you have anything after 5.4 ? That should work.

soumyadipghosh commented 5 years ago

That explains it. We have 9.1.0 but for some reason, it is defaulting to 4.8.5. Let me see how to fix it.

pratikvn commented 5 years ago

Yeah, cmake uses cc and CC for the c and c++ compilers. If gcc --version gives you 9.1.0, then you can use -DCMAKE_C_COMPILER=gcc and -DCMAKE_CXX_COMPILER=g++ as arguments to the cmake command to set these as your compilers.

soumyadipghosh commented 5 years ago

Okay, worked now! I need admin privileges to install. Have to talk to our admin as it is not letting me do sudo make install.

pratikvn commented 5 years ago

No, sudo make install is only for the default install path which is /usr/local by default. You should install it to some custom location you want like in your home directory or something like that. You can do that by passing -DCMAKE_INSTALL_PREFIX=/your/custom/path to the cmake command, and then you can do make install and that will install the ginkgo headers and the libraries to /your/custom/path and that will be your Ginkgo_DIR when you use the schwarz-lib

pratikvn commented 5 years ago

Additionally, I think you should use the develop branch instead ? You will get errors if you compile schwarz-lib because it uses the triangular solver from Ginkgo and master does not have the triangular solvers.

soumyadipghosh commented 5 years ago

Ok, anyway I have to build again.

soumyadipghosh commented 5 years ago

Alright, install worked. So I set Gingko_DIR while specifying options in the cmake here ?

Also for Metis, there is a setting called IDXTYPEWIDTH. Should I set it to 32 or 64 ?

pratikvn commented 5 years ago

Currently, I have support for only int32, but I will probably change that to automatically detect the IDXTYPEWIDTH soon. But for now, you can set it to 32. That should work.

Additionally, can you make your event-based branch from master ? You can either delete the current branch, or probably just leave it there and create another branch called event-based-comm or something like that. If you are doing this on the github page, then, first change the tab under branches to master first and then create the branch from it.

Sorry that you have to do this because I added some extra stuff into develop which is currently only in a private ginkgo repo and hence if you compile this with the normal ginkgo, it will fail.

soumyadipghosh commented 5 years ago

Ok, I will do accordingly for event-based. But before that, I am getting the following errors when I try to build schwarz-lib. CMake Error at CMakeLists.txt:24 (message): Please specify where CHOLMOD has been installed.

CMake Error at CMakeLists.txt:59 (enable_language): No CMAKE_CUDA_COMPILER could be found.

I installed Gingko and Metis successfully. I also set the environment variables for installation (I am using csh shell). I loaded mpich and boost.

pratikvn commented 5 years ago

Yes, by default, CHOLMOD is enabled. You can pass -DSCHWARZ_BUILD_CHOLMOD=off and additionally, as you are using mpich you probably do not have hwloc the as well (Unless there is a module available, or you have installed it separately). So also pass -DSCHWARZ_HAVE_HWLOC=off to the cmake command to switch that off.

There was a small issue with CUDA, so i have corrected it now and pushed it into master. Please do a git pull again. And as you probably dont want CUDA, atleast not yet, you should also pass -DSCHWARZ_BUILD_CUDA=off to switch CUDA compilation off. In case you do want to have CUDA support, then you can load the cuda module and let -DSCHWARZ_BUILD_CUDA=on as it is by default.

soumyadipghosh commented 5 years ago

I did git pull but I still get a CUDA error. This is the command I execute

cmake -G "Unix Makefiles" -DCMAKE_C_COMPILER=/opt/crc/g/gcc/8.3.0/bin/gcc -DCMAKE_CXX_COMPILER=/opt/crc/g/gcc/8.3.0/bin/g++ -DSCHWARZ_BUILD_CHOLMOD=OFF -DSCHWARZ_WITH_HWLOC=OFF DSCHWARZ_BUILD_CUDA=OFF ..

Output ``` -- The C compiler identification is GNU 8.3.0 -- The CXX compiler identification is GNU 8.3.0 -- Check for working C compiler: /opt/crc/g/gcc/8.3.0/bin/gcc -- Check for working C compiler: /opt/crc/g/gcc/8.3.0/bin/gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /opt/crc/g/gcc/8.3.0/bin/g++ -- Check for working CXX compiler: /opt/crc/g/gcc/8.3.0/bin/g++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- The CUDA compiler identification is unknown CMake Error at CMakeLists.txt:59 (enable_language): No CMAKE_CUDA_COMPILER could be found. Tell CMake where to find the compiler by setting either the environment variable "CUDACXX" or the CMake cache entry CMAKE_CUDA_COMPILER to the full path to the compiler, or to the compiler name if it is in the PATH. -- Configuring incomplete, errors occurred! See also "/afs/crc.nd.edu/user/s/sghosh2/Public/schwarz-lib/build/CMakeFiles/CMakeOutput.log". See also "/afs/crc.nd.edu/user/s/sghosh2/Public/schwarz-lib/build/CMakeFiles/CMakeError.log". ```
pratikvn commented 5 years ago

The flag is -DSCHWARZ_BUILD_CUDA not DSCHWARZ_BUILD_CUDA. You are missing the - (hypen)

soumyadipghosh commented 5 years ago

Oops, that was silly! Everything worked now. So I am guessing I have to compile bench_ras.cpp now? I did mpicxx bench_ras.cpp but it threw an error: fatal error: gflags/gflags.h: No such file or directory.

pratikvn commented 5 years ago

No, bench_ras.cpp should already be compiled. There should be an executable in build/benchmarking called bench_ras. This is the executable you run.

If you see something like [100%] Built target bench_ras, it means that the executable has been compiled. You can see how to run this exectuable in BENCHMARKING.md For example, to run for the iterative CG solver with Ginkgo and the naive 1D partition, for twosided (synchronous), doing(assuming you are in the build directory):

mpiexec -np {NUM_PROCS} ./benchmarking/bench_ras --num_iters=500 --explicit_laplacian --set_1d_laplacian_size=64 --set_tol=1e-6 --local_tol=1e-12 --partition=naive --local_solver=iterative_ginkgo

should work. Of course, if your cluster uses sbatch/srun or any of the batch submission systems, then you should use that instead of mpiexec.

See the meaning of the flags in BENCHMARKING.md. Add an additional --enable_onesided to run the asynchronous version.

soumyadipghosh commented 5 years ago

It worked! Let me play around with the code now.