Location of a line of source code for reading the number of nodes or cores from console

[Bakh]

The command

mpirun -np 4 flow --parameter-file=path_to_data/CASENAME.param

starts OPM Flow on four nodes or cores when running OPM Flow from the command line. Could anyone please help me find the location of a line of source code for reading the input (number of nodes or cores) from console?

[akva2]

uhm, depends on your mpi implementation. that's all handled by the mpirun command, which is not part of opm.

[Bakh]

4

Thank you for your reply. I believe that the number '4' in the example above is chosen by the specific user. I thought that, initially, this number is assigned as a parameter in the code (for example, I've seen NUM_PROCS, NP, MPI_PROCS, BASE_MPI_PROCS, TEST_MPI_PROCS, nprocs).

[akva2]

yes. I suggest you read some basic MPI tutorial, https://mpitutorial.com/tutorials/mpi-hello-world/ was my first hit on google, no quality control but i'm sure it's on topic.

https://github.com/OPM/opm-simulators/blob/master/opm/simulators/flow/Main.cpp#L125 is the setup code for the flow simulators. the gist of it is; mpirun parses the command line parameters, launches the specified amount of flow executables, the processes then read out the total number of processes (MPI_Comm_size) and its assigned rank (MPI_Comm_rank) from the MPI library. These mpi calls are wrapped in the Parallel::Communication class, ie. comm.size() and comm.rank();

[Bakh]

Your comments and suggestions are highly appreciated. I am new in MPI and C++ OOP. My background is Fortran. When I run the Flow executable file, it uses 1 MPI processes with 2 OMP threads on each. Are these numbers set by default in the source code? I would like to continue working with the Flow executable file and I'm interested in implementing the read a number of MPI processes from an input file.

[akva2]

threads are set by default in the executable, but that is not an mpi thing, that's an openmp thing, see https://github.com/OPM/opm-simulators/blob/master/opm/simulators/flow/Main.hpp#L669

but it's important to understand that threads and processes are different things. a process is a separate instance of the application, a thread is the number of cores usable by a single instance of the application.

when you run the executable directly only a single instance of the executable is started - it's exactly like running a normal sequential program. which means the world comm will have size 1, and the single instance will have rank 0.

when you run through mpirun -np x, x instances of the application is started. each of these processes will have y threads (defaults to 2).

pointers to read more: threads is 'shared memory parallelism', while mpi/processes are 'distributed memory parallelism'.

[akva2]

oh, and you can use both mpi and openmp in fortran applications as well.

[akva2]

and no, you cannot implement reading the number of processes from an input file in flow itself. mpirun is necessary to launch applications, and this launches the flow executable. by the time the simulator is launched it's too late. this needs to be done in a frontend script/tool.

[Bakh]

Thank you for your informative reply. In this case I need to develop a frontend script/tool which reads a number of MPI processes from an input file and launches the Flow executable file according to the input number of MPI processes. Do you think it's possible? If so, I would prefer using C++ for a frontend script/tool development.

[akva2]

it's most definitely possible. personally i'd use something more suited for such tasks though, such as python, but to each his own.

[akva2]

also maybe https://github.com/OPM/opm-utilities/tree/master/opmrun might be of use

[Bakh]

I can see that OPMRUN is an excellent tool. When developing a frontend script/tool for my purpose, may there be a need for introducing any change in the OPM Flow source code which is compiled to an executable file?

[akva2]

nope.

[Bakh]

Thanks.