Using `std::reduce` causes intel compiler to not work

[amartyads]

The newly added StaticIrregDomainDecomposition.cpp uses std::reduce, something that causes compilation to fail when compiling with icpx on HSUper. The compilation succeeds, however, if user does module load gcc before compilation. This is probably because the LD_LIBRARY_PATH gets populated with libraries from gcc, and then the compiler can find std::reduce.

Changing reduce to accumulate can be an easy fix, and since the operation is performed only once, it shouldn't be too heavy.

@HomesGH and @JakNiem initially found this bug on HSUper.

Bug replicated on HSUper icpx version: Intel(R) oneAPI DPC++/C++ Compiler 2022.1.0 (2022.1.0.20220316) default gcc version: g++ (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10) gcc version after ml gcc: g++ (Spack GCC) 12.1.0

[FG-TUM]

This is IMHO an issue (aka feature) with the module system. Some compilers like icpx rely on the libraries of the currently loaded gcc.

Possible actions we can take:

• [x] Ask cluster admins to issue relevant warning messages when loading such modules
• [x] Mention this edge case somewhere in our documentation.

[cniethammer]

ls1 requires C++17, so users should ensure, that their system and software are set up to support this. As @FG-TUM mentioned Intel uses an external C++ library so this has to be available on the system in an appropriate version and made visible to the Intel compiler for linking. See the Intel oneAPI documentation for system requirements on this https://www.intel.com/content/www/us/en/developer/articles/system-requirements/intel-oneapi-dpcpp-system-requirements.html#inpage-nav-2-undefined-1

[HomesGH]

In addition to your mentioned actions, should we still change reduce to accumulate in this case?

[HomesGH]

I have added a note to the README file, see PR #295 (see commit).

@amartyads Have you talked to the admins of HSUper regarding this issue?

[amartyads]

I have talked to the admins, here's the update:

1. You can load a newer version of the intel compilers with ml USER-SPACK/0.21.1 intel-oneapi-compilers
2. std::reduce doesn't work with the new compiler either
3. The admins are okay with adding a message saying "All c++17 features may not be supported while using the intel oneapi compiler" when ml compiler is done by user
4. If you write a simple test program with just a single std::reduce like so:

   #include <numeric>
   #include <vector>
   #include <iostream>
   int main()
   {
       const std::vector<int> v(10,100);
       const int total = std::reduce(v.begin(), v.end(), 0);
       std::cout << total << std::endl;
       const int total2 = std::accumulate(v.begin(), v.end(), 0);
       std::cout << total2 << std::endl;
       return 0;
   }

   This does not compile with icpx even with ml gcc, however ls1 still compiles. So the workaround isn't universal and something else is going on.

[cniethammer]

Issues were related to the software stack on the system and could be resolved.