Rework AMG experiments - Githubissues

pearce8 commented 1 month ago

Types of experiments

[x] The current Problem 1 experiment should be named "throughput" (varying the size of the problem, keeping resources the same). x-axis label: problem size. Add plots here.
[x] Add a strong scaling experiment. x-axis label: number of GPUs. Add plots here.
[x] Write a weak scaling experiment. x-axis label: number of GPUs. Add plots here.
[x] Add corresponding experiments for Problem 2 (throughput, strong, weak)

What is working:

[x] CUDA on lassen
[x] ROCM on tioga
[x] OpenMP on ruby: throughput works, need params for strong and weak

Thionazin commented 3 weeks ago

amg_p2_omp_weak_tot amg_p2_omp_weak_perc amg_p2_omp_strong_tot amg_p2_omp_strong_perc amg_p1_omp_weak_tot amg_p1_omp_weak_perc amg_p1_omp_strong_tot amg_p1_omp_strong_perc

Thionazin commented 3 weeks ago

amg_p2_rocm_weak_tot amg_p2_rocm_weak_perc amg_p2_rocm_strong_tot amg_p2_rocm_strong_perc amg_p1_rocm_weak_tot amg_p1_rocm_weak_perc amg_p1_rocm_strong_tot amg_p1_rocm_strong_perc

Thionazin commented 3 weeks ago

amg_p2_cuda_weak_tot amg_p2_cuda_weak_perc amg_p2_cuda_strong_tot amg_p2_cuda_strong_perc amg_p1_cuda_weak_tot amg_p1_cuda_weak_perc amg_p1_cuda_strong_tot amg_p1_cuda_strong_perc

pearce8 commented 3 weeks ago

@rfhaque Please take a look at how this PR gets around the questions, but we need a longer-term plan.

GTL: GTL is GPU Transport Layer, an HPE library required for CUDA-aware MPI.

[ ] Currently using GTL everywhere. Is this correct? [ ] We are not using CrayMPI on lassen - should we use the GTL or non-GTL version? [ ] What version should we use on Pascal (also no CrayMPI) [ ] Rocm experiments: do we need GTL? Make them match the Cuda experiments

rfhaque commented 2 weeks ago

@rfhaque Please take a look at how this PR gets around the questions, but we need a longer-term plan.

GTL: GTL is GPU Transport Layer, an HPE library required for CUDA-aware MPI [ ] Currently using GTL everywhere. Is this correct? [ ]

We are using gtl on tioga. I think we should use whichever is the default MPI version on each platform (and remove all references to gtl from ramble.yaml

We are not using CrayMPI on lassen - should we use the GTL or non-GTL version? [ ]

We use spectrum-mpi (non-gtl) which is the default mpi on lassen

What version should we use on Pascal (also no CrayMPI) [ ]

We use mvapich2-2.3.7-gcc-11.2.1 on Pascal, that is not gtl enabled

Rocm experiments: do we need GTL? Make them match the Cuda experiments

We do not need gtl in the rocm experiments. We can use the default mode provided by the underlying MPI implementation on the platform

LLNL / benchpark

Rework AMG experiments #309