ylikx / forpy

Forpy - use Python from Fortran
GNU Lesser General Public License v3.0
214 stars 51 forks source link

multiple initialize/finalize? #10

Open mgovoni-devel opened 5 years ago

mgovoni-devel commented 5 years ago

Is it possible to have multiple initialize/finalize statements? It would be nice to confine forpy to given regions.

ylikx commented 5 years ago

There is an issue, when calling first forpy_finalize and forpy_initialize afterwards and it is related to numpy. By default forpy imports numpy in forpy_initialize. Numpy has a problem when it is re-initialized and crashes. (It should work when you use forpy_initialize(use_numpy=.false.)) Besides numpy, there could also be other Python C extensions with a similar issue.

See also this Python C API related discussion on Stackoverflow: https://stackoverflow.com/questions/7676314/py-initialize-py-finalize-not-working-twice-with-numpy

tonino102008 commented 4 years ago

How could I solve this problem?? Maybe a check on wheter forpy is initialized, to avoid calling it for the second time, could be useful? I have this issue because I'm using it in a program that can run in parallel and forpy seems to crash due to a numpy issue...

ylikx commented 4 years ago

I would suggest to initialize forpy as early as possible in your (main) program and finalize only at the very end of the main program, e. g.:

program my_program
use forpy_mod
implicit none

integer :: ierror
ierror = forpy_initialize()

! ....
! do stuff
! ...

call forpy_finalize
end program

Do not use forpy_initialize/forpy_finalize pairs in loops or subroutines that you call more than once.

Regarding parallelism: Multiprocessing (e.g. with MPI) should not be a problem as each process can have its own Python interpreter.

Multithreading (e. g. with OpenMP) is problematic, since a thread has to hold the Python GIL (Global interpreter lock) when doing any Python stuff. There are Python C API functions to deal with that but there is no interface for them in forpy (yet).

See also: https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock