WIP: Proof of concept

[HenningScheufler]

The goal of this branch is to quickly to reach a proof of concept for the DSL. The implementation and the planning of the library with accelerators in not straightforward and lots of approach, we used in the past are not going not work. But we don't know which one. Therefore, de-risking the implementation and gaining experience with hybrid architecture is key to write a maintainable, user-friendly and performant new core. The overall approach is to write unit tests and benchmarks to enable quick design iterations. General Approach: ExplicitOperatores -> ImplicitOperators -> Parallelization

The following implementation are required (TBD):

• [x] documentation
• [x] basic blas
• [ ] unstructuredMesh
  • [ ] mesh readers
  • [ ] mesh writers
• [ ] patch-base and tree-based (AMREX and t8code) (mainly to test the flexibilty of the architecture for now)
• [ ] stencils
• [ ] operators
  • [ ] implicit
  • [ ] explicit
• [ ] post processing (python bindings) simply testing
  • [ ] also has to work with the other FVM approaches
• [ ] domain fields (internal + boundaryfields (Foam --> geometricFields) )
  • [ ] strategy pattern on GPUs
• [ ] sparse matrix solvers
• [ ] DSL

@greole @bevanwsjones

@greole can you port this repo to exasim and update the build process?

https://github.com/HenningScheufler/NeoFOAM_GPL/tree/proofofconcept

NOTE everything that is GPL needs be stored NeoFOAM_GPL:

• OpenFOAM
• t8code

So we have to work on both repos

[HenningScheufler]

Ideally Kokkos would be in the build process. Currently auto complete does not work

[HenningScheufler]

Another point is how to split header and source with CUDA with g++ or clang

[bevanwsjones]

Solved: On the Nivida install page the export path commands still point to cuda 12.2 where as for me the install was cuda 12.3, setting to the correct path fixed the issue.

when I run the kokkos install script I get:

.
.
.
/home/bevan/Applications/NeoFOAM/kokkos-4.2.00/bin/nvcc_wrapper: line 658: nvcc: command not found
CMake Error at cmake/kokkos_compiler_id.cmake:12 (STRING):
  STRING sub-command REPLACE requires at least four arguments.
Call Stack (most recent call first):
  cmake/kokkos_compiler_id.cmake:56 (kokkos_internal_have_compiler_nvcc)
  cmake/kokkos_tribits.cmake:175 (INCLUDE)
  CMakeLists.txt:226 (KOKKOS_SETUP_BUILD_ENVIRONMENT)

-- Using -std=c++17 for C++17 standard as feature
CMake Error at cmake/kokkos_test_cxx_std.cmake:127 (MESSAGE):
  Invalid compiler for CUDA.  The compiler must be nvcc_wrapper or Clang or
  NVC++ or use kokkos_launch_compiler, but compiler ID was GNU
Call Stack (most recent call first):
  cmake/kokkos_tribits.cmake:203 (INCLUDE)
  CMakeLists.txt:226 (KOKKOS_SETUP_BUILD_ENVIRONMENT)

Any ideas what the issue is?

[HenningScheufler]

Clang should be cuda compatible and could be worth a try?

I use gcc the default of ubuntu 22.04 and the cuda base compiler:

and used the following variable in the bashrc

export CUDA_HOME=/usr/local/cuda
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
export PATH=$PATH:$CUDA_HOME/bin

[bevanwsjones]

@HenningScheufler, when you get a minute would you add a 'quick and dirty' install for OpenFoam, just so I can get everything here to compile?

Good news my laptop has an Nvidia card :)

[HenningScheufler]

cd $HOME
mkdir OpenFOAM
# https://develop.openfoam.com/Development/openfoam/-/blob/master/doc/Build.md
wget https://dl.openfoam.com/source/v2306/OpenFOAM-v2306.tgz
wget https://dl.openfoam.com/source/v2306/ThirdParty-v2306.tgz
tar zxvf OpenFOAM-v2306.tgz 
tar zxvf ThirdParty-v2306.tgz
cd OpenFOAM-v2306
source etc/bashrc
# needs quite a bit of RAM
./Allwmake -j 4 -q -l # -j 4 for 4 cores, -q for quiet, -l for log 

# add source $HOME/OpenFOAM/OpenFOAM-v2306/etc/bashrc to .bashrc

[HenningScheufler]

I added sphinx with breathe as documentation tool. The documentation feature of github copilot works pretty well to get a good first draft.