project-asgard / asgard

MIT License
27 stars 20 forks source link

time-advance-tests with scalapack solver are failing #691

Open quantumsteve opened 4 months ago

quantumsteve commented 4 months ago

Describe the bug A clear and concise description of what the bug is.

ctest -R time_advance-test-mpi_4 --output-on-failure

time_advance-tests: /home/svh/Documents/asgard/src/asgard_vector.hpp:1084: asgard::fk::vector<P, mem, resrc>::vector(const asgard::fk::vector<P, omem, resrc>&, int, int, bool) [with asgard::mem_type m_ = asgard::mem_type::view; <template-parameter-2-2> = void; asgard::mem_type omem = asgard::mem_type::owner; P = float; asgard::mem_type mem = asgard::mem_type::view; asgard::resource resrc = asgard::resource::host]: Assertion `stop_index < vec.size()' failed.
time_advance-tests: /home/svh/Documents/asgard/src/asgard_vector.hpp:1084: asgard::fk::vector<P, mem, resrc>::vector(const asgard::fk::vector<P, omem, resrc>&, int, int, bool) [with asgard::mem_type m_ = asgard::mem_type::const_view; <template-parameter-2-2> = void; asgard::mem_type omem = asgard::mem_type::owner; P = float; asgard::mem_type mem = asgard::mem_type::const_view; asgard::resource resrc = asgard::resource::host]: Assertion `stop_index < vec.size()' failed.
time_advance-tests: /home/svh/Documents/asgard/src/asgard_vector.hpp:1084: asgard::fk::vector<P, mem, resrc>::vector(const asgard::fk::vector<P, omem, resrc>&, int, int, bool) [with asgard::mem_type m_ = asgard::mem_type::const_view; <template-parameter-2-2> = void; asgard::mem_type omem = asgard::mem_type::owner; P = float; asgard::mem_type mem = asgard::mem_type::const_view; asgard::resource resrc = asgard::resource::host]: Assertion `stop_index < vec.size()' failed.
time_advance-tests: /home/svh/Documents/asgard/src/asgard_vector.hpp:1084: asgard::fk::vector<P, mem, resrc>::vector(const asgard::fk::vector<P, omem, resrc>&, int, int, bool) [with asgard::mem_type m_ = asgard::mem_type::view; <template-parameter-2-2> = void; asgard::mem_type omem = asgard::mem_type::owner; P = float; asgard::mem_type mem = asgard::mem_type::view; asgard::resource resrc = asgard::resource::host]: Assertion `stop_index < vec.size()' failed.

To Reproduce Steps to reproduce the behavior:

  1. git commit hash being built

def3174fd42a1a0010b9286a4f3c1afd308db72b

  1. cmake command

cmake ../ -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++ -DASGARD_USE_MPI=1 -DASGARD_USE_SCALAPACK=1 -DASGARD_RECOMMENDED_DEFAULTS=1

  1. full program/test invocation command

ctest -R time_advance-test-mpi_4 --output-on-failure

  1. additional steps

Expected behavior A clear and concise description of what you expected to happen.

System:

AWS CodeBuild - asgard-unit-mpi-gxx-scalapack

none

Additional context Add any other context about the problem here.

Found in #685. Tests disabled here

mkstoyanov commented 4 months ago

This may or may not be an easy fix but it raises a question to discuss in the next meeting. Should we continue supporting SCALAPACK? Given how much more expensive it is to form the Kronecker matrices, even compared to the local kronmult, perhaps we should focus efforts on GMRES+MPI for local kronmult instead.

Let's discuss on the next meeting.