Running gipc on FreeBSD fails

jgehrcke commented 10 years ago

Originally reported by: bra (Bitbucket: bra, GitHub: bra)

Hi,

Running the serverclient.py on FreeBSD 10 (gevent 1.0, python 2.7.6 - it also fails with the same on FreeBSD 9 and python 2.7.3, so it seems it's nothing new) gives:

#!python
# python serverclient.py 
Process _GProcess-1:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/local/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python2.7/site-packages/gipc/gipc.py", line 281, in _child
    _reset_signal_handlers()
  File "/usr/local/lib/python2.7/site-packages/gipc/gipc.py", line 873, in _reset_signal_handlers
    signal.signal(s, signal.SIG_DFL)
ValueError: signal number out of range

>>> for s in [s for s in dir(signal) if s.startswith("SIG")]:
...     print s, getattr(signal, s)
... 
SIGABRT 6
SIGALRM 14
SIGBUS 10
SIGCHLD 20
SIGCONT 19
SIGEMT 7
SIGFPE 8
SIGHUP 1
SIGILL 4
SIGINFO 29
SIGINT 2
SIGIO 23
SIGIOT 6
SIGKILL 9
SIGPIPE 13
SIGPROF 27
SIGQUIT 3
SIGRTMAX 126
SIGRTMIN 65
SIGSEGV 11
SIGSTOP 17
SIGSYS 12
SIGTERM 15
SIGTRAP 5
SIGTSTP 18
SIGTTIN 21
SIGTTOU 22
SIGURG 16
SIGUSR1 30
SIGUSR2 31
SIGVTALRM 26
SIGWINCH 28
SIGXCPU 24
SIGXFSZ 25
SIG_DFL 0
SIG_IGN 1

Using this:

#!python
def _reset_signal_handlers():
    for s in _signals_to_reset:
        try:
            signal.signal(s, signal.SIG_DFL)
        except ValueError:
            print "SIG_DFL failed for",s

yields:

#!python
# python serverclient.py
SIG_DFL failed for 65
SIG_DFL failed for 126
1000 clients served within 0.42 s.

Bitbucket: https://bitbucket.org/jgehrcke/gipc/issue/10

jgehrcke commented 10 years ago

Original comment by bra (Bitbucket: bra, GitHub: bra):

Sure, thanks!

jgehrcke commented 10 years ago

Original comment by Jan-Philip Gehrcke (Bitbucket: jgehrcke, GitHub: jgehrcke):

Fix issue #10: check signal number against NSIG.

jgehrcke commented 10 years ago

Original comment by Jan-Philip Gehrcke (Bitbucket: jgehrcke, GitHub: jgehrcke):

It's not really about 'this method for messing with signals'. This fails on FreeBSD:

>>> signal.signal(signal.SIGRTMAX, lambda *a: None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: signal number out of range

No feedback so far in http://bugs.python.org/issue20584. For fixing this in gipc, I would just add a manual check for the error condition. Agree?

jgehrcke commented 10 years ago

Original comment by Jan-Philip Gehrcke (Bitbucket: jgehrcke, GitHub: jgehrcke):

Regarding gipc, I think just ignoring a ValueError in signal.signal(s, signal.SIG_DFL) would be a proper solution. What do you think?

Background:

SIGRTMIN/MAX signals were added to FreeBSD's signal.h in 2005: http://svnweb.freebsd.org/base/head/sys/sys/signal.h?r1=149337&r2=151306

They were officially supported from FreeBSD 7.0 on: http://www.freebsd.org/cgi/query-pr.cgi?pr=99517

They are still there now, with the same numerical values: http://svnweb.freebsd.org/base/head/sys/sys/signal.h?revision=233519&view=markup#l117

NSIG is defined in the signal.h, and your Python's signal.NSIG most likely is defined as 32: http://svnweb.freebsd.org/base/head/sys/sys/signal.h?revision=233519&view=markup#l331 Can you have a look? I am not sure about __BSD_VISIBLE (see http://monkey.org/freebsd/archive/freebsd-newbies/200501/msg00118.html)

Depending on __BSD_VISIBLE, your Python's signal.NSIG either is 32 or 64, and both are obviously not prepared for the SIGRTMIN/MAX signals. On my Linux NSIG is 65 and the SIGRTMIN/MAX values are below that threshold:

>>> signal.NSIG
65
>>> signal.SIGRTMAX
64
>>> signal.SIGRTMIN
34

The first realtime-related signal on FreeBSD is actually 33: http://svnweb.freebsd.org/base/head/sys/sys/signal.h?revision=233519&view=markup#l114

Two things are to be clarified, I think:

Is NSIG on FreeBSD meaningful and -- if yes -- did the FreeBSD guys purposely not include the realtime related stuff in the interval as given by NSIG?
There seems to be an inconsistency in Pythons signalmodule.c. On the one hand, there is NSIG, which might even be just a guess. On the other hand, there can be signals with integer values higher than NSIG. Im my opinion, NSIG should never be guessed, but then rather be retrieved in a max(signalvalues) fashion. If the system provides NSIG and this is not the 'true' maximum, then Python is indeed not to blame.

jgehrcke commented 10 years ago

Original comment by bra (Bitbucket: bra, GitHub: bra):

Of course, it's built on the same system I use it on (from ports). BTW, here's the cause: FreeBSD doesn't have _NSIG, so NSIG will be 64, and on line number 300, it will raise the above exception, because there are two signals above this value (65 and 126). It seems nobody uses this method for messing with signals, at least on FreeBSD and with Python. :)

jgehrcke commented 10 years ago

Original comment by Jan-Philip Gehrcke (Bitbucket: jgehrcke, GitHub: jgehrcke):

Thanks for reporting, this is interesting. I will have a deeper look soon. Until then, can you please check if the Python versions you are using have actually been built on or at least specifically for the system you are using them on? I am asking because Python's signal module is built upon compile time from https://github.com/python/cpython/blob/2.7/Modules/signalmodule.c, which includes the system's signal header file via #include <signal.h>. In principle, all signal codes available to Python's signal module should therefore be valid ones. It could however be that on FreeBSD the default signal action is not allowed to be set for certain signals -- I will have to check the man pages. A workaround is quite simple to implement, but I would like to understand this issue thoroughly before taking action.

jgehrcke / gipc

Running gipc on FreeBSD fails #10

Background: