STOP command kills entire python instance

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Run a Fortran program wrapped using f2py in Python; any STOP command in 
Fortran will kill the entire Python instance

What version of the product are you using? On what operating system?
Version 2

Please provide any additional information below.

I am trying to wrap a large, existing Fortran program 
(http://www.itp.uzh.ch/~teyssier/ramses/RAMSES.html) with multiple files and 
modules in f2py and call it from a Python package. The Fortran program makes 
liberal use of STOP commands in both upper and lower case and is under active 
development, so removing the STOP commands manually will be a pain. I would 
like to keep the Python instance active after the Fortran program ends, rather 
than having the Fortran program kill the Python script upon exit. Putting the 
f2py call in a separate thread doesn't appear to help; a STOP command in a 
spawned thread will kill the Python instance that spawned it. If you can 
suggest work-arounds that don't involve calling the program from the command 
line, let me know!

Original issue reported on code.google.com by samg...@googlemail.com on 8 Nov 2013 at 3:46

[GoogleCodeExporter]

Note that in Fortran STOP is a statement of the language that executes system 
level function. As a result, it is impossible for f2py catch the results of 
STOP execution.

I would write a script that walks through the source code of the Fortran 
program and replace all STOP statements with something more appropriate, e.g. a 
special function call that records the STOP statement argument and makes a 
longjmp call to the wrapper function, all necessary modifications to the 
generated wrapper function, eg defining longjmp points, etc. are supported by 
f2py signature file specifications.

In principle, fparser could be used for that but it could be overkill as a 
simple python script should be sufficient for replacing STOP statements with 
something other.

Original comment by pearu.peterson on 10 Nov 2013 at 10:49

[GoogleCodeExporter]

Ah, I see, that makes sense. Thinking about it, it shouldn't be too hard to add 
a simple script before compilation that replaces all the STOP commands. Thanks 
for your advice - I'll let you know if I have any other problems with this. My 
only other idea was to spawn a separate Python instance with the Fortran code 
in it so that if it goes down the main thread stays alive, but I can't think of 
a clean way to do this short of calling Python with os.system() or similar.

Original comment by samg...@googlemail.com on 10 Nov 2013 at 2:30

[GoogleCodeExporter]

I wrote some code to skim the Fortran and replace the STOP calls with something 
that references longjmp (and isetjmp), and this works, but apparently gfortran 
can't find longjmp/isetjmp ("undefined reference to `longjmp_' / collect2: 
error: ld returned 1 exit status"). It's possible it's been removed: 
http://gcc.gnu.org/ml/fortran/2010-11/msg00009.html - I'll keep poking, but it 
seems quite difficult to find meaningful documentation on longjmp in gfortran 
in any case.

This stackoverflow seems to propose a working C++ wrapper that calls longjmp in 
C++, meaning you don't need to modify the Fortran at all: 
http://stackoverflow.com/questions/19596375/intercepting-fortran-stop-from-c 
(no idea how stable or portable this fix is, of course). I'd rather not go to 
the trouble of wrapping Fortran in C++ and then wrapping that in Python, of 
course, but if it works...

Original comment by samg...@googlemail.com on 20 Nov 2013 at 1:42

[GoogleCodeExporter]

1) I don't think that you need to deal with longjmp in Fortran. You can replace 
Fortran STOP statement with a call to C function that will deal with the 
longjmp stuff.

2) Finding the exit function as described in intercepting-fortran-stop-from-c 
is equivalent to replacing STOP statement. You would not need to change Fortran 
code at the expense of loosing portability (is portability relevant in your 
case?). Otherwise, longjmp stuff has to be implemented anyway.

Original comment by pearu.peterson on 20 Nov 2013 at 1:52

[GoogleCodeExporter]

OK, it works, thanks! It's a bit convoluted but seems to work. For 
reference/posterity, my solution was the following (I've changed around some of 
the names and simplified my code a bit, but it should work as written):
1) Replace every "STOP" call with "call long_jump" using a Python script (I can 
post this up if you like; I jumped through a few hoops to get rid of stuff like 
comments/strings including the characters "stop" or functions called 
"stop_stuff", etc.
2) Write these files:
---
clean_stop.c:
#include <stdio.h>
#include <setjmp.h>

static jmp_buf buf;

void run_fortran_();

void long_jump_()
{
  longjmp(buf,1);
}

int set_jump_()
{
  if(!setjmp(buf))
  {
    run_fortran_();
  }
}
---
myfortran.f90:
program myfortran
  ! This is optional; allows you to run it as a command-line program too
  call set_jump
end program myfortran

subroutine run
  ! This is what you call in Python
  call set_jump
end subroutine run

subroutine run_fortran
  ! FORTAN CODE GOES HERE

end subroutine run_fortran
---
3) Compile both as object files in gfortran and gcc
4) Run "f2py -m myfortran -h myfortran.pyf myfortran.f90" (you can delete the 
interface call to run_fortran if you like, might make things cleaner for the 
users)
5) Run "f2py -lgfortran" -c myfortran.pyf *.o 
6) In Python, call "import myfortran; myfortran.run()"
(Note: the output of this program is as files, I assume returning variables 
should be a simple thing to hack into the solution above)

I suspect it might have been simpler to embed the C code in Python as the entry 
point, but never mind.

Original comment by samg...@googlemail.com on 20 Nov 2013 at 3:34

[GoogleCodeExporter]

Thanks for your feedback!

After a thought, I found perhaps an easier solution. It is described in

https://code.google.com/p/f2py/wiki/FAQ2ed

Original comment by pearu.peterson on 20 Nov 2013 at 11:04

[GoogleCodeExporter]

Ah, cunning. I might try your solution #1 if I have to revisit the problem for 
whatever reason; it seems a bit less invasive than the solution I gave above as 
that way I don't need to mess about with C or modify the Makefile. Thanks again 
for your help!

Original comment by samg...@googlemail.com on 20 Nov 2013 at 11:10

[GoogleCodeExporter]

Original comment by pearu.peterson on 20 Nov 2013 at 11:38

• Changed state: Done

[GoogleCodeExporter]

Note that I have changed the FAQ a little: actually there is no reason to 
remove STOP statements, they can be left in provided that they precede a call 
to Python function that raises an exception. In this way the Fortran sources 
will be still usable by Fortran programs that need to provide the f2pystop 
function that does nothing.

Original comment by pearu.peterson on 21 Nov 2013 at 7:42

[GoogleCodeExporter]

Sorry for dragging this back up again, but one thing I noticed was the module 
Multiprocessing: http://docs.python.org/2/library/multiprocessing.html - I 
haven't had a chance to confirm, but I believe this spawns multiple Python 
instances, so in theory if the stop command is called it should only affect its 
containing process (I found this as I didn't manage to get the !f2py threadsafe 
command to work).

Original comment by samg...@googlemail.com on 10 Dec 2013 at 10:22