search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.47k stars 487 forks source link

AlarmInterrupt in destructor #24217

Open vbraun opened 7 years ago

vbraun commented 7 years ago

I think there is a general logic error with AlarmInterrupt in that it can be raised in a destructor, where Python will then ignore it. This causes doctest timeout like:

sage: try:  # long time
    alarm(1)
    while True:
        D._save(s, fn)
except (AlarmInterrupt, OSError, AttributeError):
    # OSError could happen due to a double close() in
    # Python's tempfile module.
    # AttributeError could happen due to interrupting
    # in _TemporaryFileWrapper.__init__
    # (see https://github.com/sagemath/sage-prod/issues/22423)
    pass ## line 201 ##
Exception cysignals.signals.AlarmInterrupt: AlarmInterrupt() in <bound method _TemporaryFileWrapper.__del__ of <closed file '<fdopen>', mode 'w+b' at 0xc8023390>> ignored
------------------------------------------------------------------------
/home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x40c2)[0xf69a20c2]
/home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x411c)[0xf69a211c]
/home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x6d2f)[0xf69a4d2f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf76f2cb0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf76f2c99]
/lib/i386-linux-gnu/libpthread.so.0(fsync+0x1c)[0xf74b960c]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x13a373)[0xf7607373]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x7407)[0xf75bf067]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x6afc1)[0xf7537fc1]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x484ca)[0xf75154ca]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_CallFunctionObjArgs+0x78)[0xf7503f28]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x4ac6)[0xf75bc726]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ed4)[0xf75bdb34]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCode+0x2e)[0xf75c1e2e]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x60f0)[0xf75bdd50]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x6afc1)[0xf7537fc1]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x484ca)[0xf75154ca]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0xa8bcd)[0xf7575bcd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5c34)[0xf75bd894]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ed4)[0xf75bdb34]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x6afc1)[0xf7537fc1]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0x484ca)[0xf75154ca]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0xa9035)[0xf7576035]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(+0xa5909)[0xf7572909]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyObject_Call+0x4d)[0xf75036cd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5c34)[0xf75bd894]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5ed4)[0xf75bdb34]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5dd2)[0xf75bda32]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7fe)[0xf75c1cbe]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyEval_EvalCode+0x2e)[0xf75c1e2e]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyRun_FileExFlags+0x77)[0xf75e5457]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0xdd)[0xf75e69bd]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(PyRun_AnyFileExFlags+0x67)[0xf75e70e7]
/home/buildbot/slave/sage_git/build/local/lib/libpython2.7.so.1.0(Py_Main+0xcfd)[0xf75fdd2d]
python(main+0x27)[0x5655d617]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6)[0xf72ac276]
python(+0x655)[0x5655d655]
------------------------------------------------------------------------
Attaching gdb to process id 27022.
Cannot find gdb installed
GDB is not installed.
Install gdb for enhanced tracebacks.
------------------------------------------------------------------------
**********************************************************************
----------------------------------------------------------------------
sage -t --long local/lib/python2.7/site-packages/sagenb/storage/filesystem_storage.py  # Timed out
----------------------------------------------------------------------

Not sure what the right solution is, though. Ideally, the signal would be delivered as soon as the dtor is completed.

CC: @jdemeyer

Component: c_lib

Issue created by migration from https://trac.sagemath.org/ticket/24217

jdemeyer commented 7 years ago

Replying to @vbraun:

Python will then ignore it.

There is nothing that cysignals can do about that. I consider it a Python bug: if __del__ can't handle exceptions, it shouldn't check for signals either. See https://bugs.python.org/issue31388

I don't see a solution without patching Python somehow.

jdemeyer commented 6 years ago
comment:2

It would be convenient if Python somehow had a hook to deal with unraisable exceptions. But it doesn't: it just deletes the exception and there is (AFAIK) no way to figure out that it did that.

nbruin commented 6 years ago
comment:3

One approach taken by some multi-threaded applications (ECL comes to mind) is that the main thread is dedicated to handling asynchronous signals and work is done in other threads. In principle, it might be possible to do something similar in sage (probably with just 2 threads in total, normally). Then the signal handling thread can set up all kinds of flags that other threads can respond to whenever they are ready. This "cure" may be worse than the problem, though.

jdemeyer commented 6 years ago
comment:4

Replying to @nbruin:

One approach taken by some multi-threaded applications (ECL comes to mind) is that the main thread is dedicated to handling asynchronous signals and work is done in other threads.

The problem is not signal handling, the problem is that Python throws away exceptions.

nbruin commented 6 years ago
comment:5

Replying to @jdemeyer:

The problem is not signal handling, the problem is that Python throws away exceptions.

That's one way of looking at it, but it seems that the exceptions that are really problematic when they're thrown away are the ones stemming from asynchronous signals; at least that's the kind that triggers the reported behaviour in the ticket. It may be the case that splitting threads allows you to work around the behaviour in python without patching it.

Personally I think I can live with knowing that I shouldn't fully trust results from a sage session that has experienced an interruption. That would be my default assumption for computational software anyway. I'm not really advocating that we explore this option, but it seems a possibility.

jdemeyer commented 6 years ago
comment:6

Replying to @nbruin:

One approach taken by some multi-threaded applications (ECL comes to mind) is that the main thread is dedicated to handling asynchronous signals and work is done in other threads. In principle, it might be possible to do something similar in sage (probably with just 2 threads in total, normally). Then the signal handling thread can set up all kinds of flags that other threads can respond to whenever they are ready.

This is actually more or less how Python internally handles interrupts. It's not literally multi-threaded, but I don't think that this matters. When an interrupt occurs, it just sets a flag that the interpreter checks at suitable times.

So two possible fixes that I see (both involve patching Python):

  1. Python should only check for signals when it can actually handle them.

  2. Python should have a mechanism to deal with unraisable exceptions: say, a callback method __unraisable__ on the exception object. That way, we can re-enable the interrupt flag when a KeyboardInterrupt exception is ignored.

I think that 2. can realistically be implemented, but I don't feel like fighting with Python upstream to make them accept that change (they are extremely conservative and it's hard to push any non-trivial changes).