ammarhakim / gkyl

This is the main source repo for the Gkeyll 2.0 code. Please see gkeyll.rtfd.io for details.
https://gkeyll.readthedocs.io/en/latest/
55 stars 15 forks source link

error running vlasov quickstart example in parallel #100

Closed cwsmith closed 1 year ago

cwsmith commented 1 year ago

Running the vlasov quickstart parallel example from here:

https://gkeyll.readthedocs.io/en/latest/quickstart/vlasovExample1.html#running-the-simulation

using the lua script here:

https://gkeyll.readthedocs.io/en/latest/quickstart/inputFiles/vm-tsw-2x2v.html

on a workstation using mpich 3.3.2 fails with the following error:

$ mpirun -np 10 gkyl vm-tsw-2x2v.lua                                                                                      
Thu Oct 06 2022 15:15:25.000000000
Gkyl built with 4c3e5680196b
Gkyl built on Sep 22 2022 12:37:06
Initializing Vlasov-Maxwell simulation ...
*** LOAD ERROR ***
 /lore/cwsmith/spaceWeather/gkylsoft/bin/Comm/Mpi.lua:389: bad argument #3 to 'MPI_Win_shared_query' (cannot convert 'unsigned int [1]' to 'int64_t *')
 <... snip ...>

On an Intel Mac we also tried letting the gkyl install scripts install OpenMPI and hit the same errors.

On the workstation, making the following changes to disable shared memory and increase the number of subdomains appeared to avoid the failures:

101,102c101,102
<    decompCuts = {1,1},                      -- Cuts in each configuration direction
<    useShared = true,                        -- If using shared memory
---
>    decompCuts = {5,2},                      -- Cuts in each configuration direction
>    useShared = false,                       -- If using shared memory
manauref commented 1 year ago

@cwsmith we are not supporting MPI shared memory anymore. Please keep useShared = false, or better yet delete specification of useShared from your input files. In the near future we will try to remove this from example input files, and in the less near future we will remove MPI shared memory from gkeyll altogether (in fact we already have, but it's in a branch we haven't merged).

If you application needs shared memory to make progress, please get in touch with one of the Gkeyll developers. They may be able to suggest workarounds.

cwsmith commented 1 year ago

OK. Thanks for the quick feedback.