reguly
commented
1 year ago Thanks for looking into this - I have also seen issues with -gpudirect on MG-CFD, even with SoA, but had no time yet to investigate..
On 2023. Apr 19., at 0:16, Toby Flynn @.***> wrote:
I'm coming across an issue when using OP2's -gpudirect flag without code generating for SOA on GPUs and I've been able to recreate the issue using the airfoil sample app (the exact one I was using is the HDF5 double precision airfoil). When running with SOA code generation, everything works fine and the test passes for airfoil but if SOA is not used the test on the residual fails. There are no segfaults or runtime errors, the app is just producing an incorrect solution as far as I can tell. I've had an initial look at the halo exchange code that supports gpudirect and I couldn't see the issue immediately but I will have a more thorough look this week.
— Reply to this email directly, view it on GitHub https://github.com/OP-DSL/OP2-Common/issues/239, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJWVVKXTQBPTO74DYL4QITXB4HCRANCNFSM6AAAAAAXDHQ45U. You are receiving this because you are subscribed to this thread.
I'm coming across an issue when using OP2's -gpudirect flag without code generating for SOA on GPUs and I've been able to recreate the issue using the airfoil sample app (the exact one I was using is the HDF5 double precision airfoil). When running with SOA code generation, everything works fine and the test passes for airfoil but if SOA is not used the test on the residual fails. There are no segfaults or runtime errors, the app is just producing an incorrect solution as far as I can tell. I've had an initial look at the halo exchange code that supports gpudirect and I couldn't see the issue immediately but I will have a more thorough look this week.