Open pdmullen opened 1 month ago
Which pointers? The functor? The class pointer? Or data pointers specific to spiner?
I think @pdmullen is refering to the pointers to data accessed. The call pattern that fails looks something like this:
// on device
Real *some_device_pointer = &my_device_view(0);
auto my_functor = [=](auto eos) {
Kokkos::parallel_for(inner_range, [=](int it) {
eos.SomeCall(pointer[i]);
});
};
eos.Evaluate(my_functor);
I think @pdmullen is refering to the pointers to data accessed. The call pattern that fails looks something like this:
// on device Real *some_device_pointer = &my_device_view(0); auto my_functor = [=](auto eos) { Kokkos::parallel_for(inner_range, [=](int it) { eos.SomeCall(pointer[i]); }); }; eos.Evaluate(my_functor);
I assume that should be a KOKKOS_LAMBDA
rather than a regular lambda?
Nope because that whole call is inside a kokkos kernel already.
Nope because that whole call is inside a kokkos kernel already.
Evaluate
isn't a PORTABLE_FUNCTION
. Does it need to be if it's being called on device?
That's why Evaluate
is marked constexpr
that's supposed to allow us to evaluate it on device. But it's possible something is going wrong there.
ahhhhhh gotcha
I am experiencing an issue in a downstream app wherein an
Evaluate
call (see https://github.com/lanl/singularity-eos/blob/dcd7a6efd5acbfa53903f08f02aca7abfa5c3f5f/singularity-eos/eos/eos_base.hpp#L135-L139) leads to data corruption on GPUs (volta
withnvhpc
). I can seemingly only encounter the issue when working withSpiner
EOS tables.Printing data at addresses from explicit
ptr
s inside and outside theEvaluate
call show differing values.@jonahm-LANL asked me to make an issue --- we will work on isolating the problem to see if this is a downstream app or
singularity
issue. If the latter, I am sure a reproducer is in order...