install default signal handlers?

bluesmoon commented 7 years ago

Environment

julia> VERSION
v"0.4.5"

julia> Pkg.status("PyCall")
 - PyCall                        1.7.2

Python 3.4.3

Prerequisites

Install snowflake-connector-python
Get a snowflake account: https://sfc-bzops-1.snowflake.net/?plan=standard (requires credit card, but first 30 days are free and you can cancel before that, you might also be able to tell them that you're working on this bug and ask for a limited test account that only has access to the sample data)

Description

After connecting to the snowflake database, the first SQL statement will always fail with a Python TypeError('signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object',)

This does not happen when running through python (see below)

julia Code

julia> using PyCall

julia> @pyimport snowflake.connector as snowflake

julia> conn = snowflake.connect(user="<your user>", password="<your password>", account ="<your account>")
PyObject <snowflake.connector.connection.SnowflakeConnection object at 0x7efc067dbbe0>

julia> cs = conn[:cursor]()
PyObject <snowflake.connector.cursor.SnowflakeCursor object at 0x7efc09eafb00>

julia> cs[:execute]("SELECT current_version()")
ERROR: PyError (:PyObject_Call) <class 'TypeError'>
TypeError('signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object',)
  File "/usr/local/lib/python3.4/dist-packages/snowflake/connector/cursor.py", line 442, in execute
    _is_put_get=_is_put_get)
  File "/usr/local/lib/python3.4/dist-packages/snowflake/connector/cursor.py", line 348, in _execute_helper
    signal.signal(signal.SIGINT, original_sigint)

 [inlined code] from /home/ubuntu/.julia/v0.4/PyCall/src/exception.jl:81
 in _pycall at /home/ubuntu/.julia/v0.4/PyCall/src/PyCall.jl:546
 in pycall at /home/ubuntu/.julia/v0.4/PyCall/src/PyCall.jl:568
 in call at /home/ubuntu/.julia/v0.4/PyCall/src/PyCall.jl:571

julia> cs[:execute]("SELECT current_version()")
PyObject <snowflake.connector.cursor.SnowflakeCursor object at 0x7efc09eafb00>

julia> row = cs[:fetchone]()
("1.76.0",)

python Code

Python 3.4.3 (default, Sep 14 2016, 12:36:27) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import snowflake.connector as snowflake
>>> conn = snowflake.connect(user="<your user>", password="<your password>", account ="<your account>")
>>> cs = conn.cursor()
>>> cs.execute("SELECT current_version()")
<snowflake.connector.cursor.SnowflakeCursor object at 0x7f2c27cb78d0>
>>> row = cs.fetchone()
>>> print(row)
('1.76.0',)
>>>

I can execute other statements after the first one fails, unless I leave the connection idle for a long time, in which case I will get the error again, and the very next statement will work without issue.

Any idea what could be wrong or if I could do anything to fix this?

Thanks,

Philip

stevengj commented 7 years ago

~~Could be #268 ... the Python code is claims it only wants a callable object, but it is actually doing something that requires a pure Python function; see that issue for a possible workaround.~~

No, you aren't actually passing any callback functions, it looks like.

stevengj commented 7 years ago

I don't know what signal handler it is referring to. It could be that the python executable registers some default signal handler, but when we load libpython as a library it doesn't do that, and the snowflake module gets confused.

stevengj commented 7 years ago

(I assume that you're using the same version of Python in PyCall as in your Python code. The Python executable corresponding to the version used in PyCall is PyCall.python)

stevengj commented 7 years ago

Okay, I looked into the snowflake-connector source code, and the above exception seems to be thrown when it calls signal.signal to restore the original signal handler after temporarily changing it.

Can you check that the following works?

using PyCall
signal = pyimport("signal")
original_sigint = signal["getsignal"](signal["SIGINT"])
signal["signal"](signal["SIGINT"], original_sigint)

bluesmoon commented 7 years ago

That did not work, I get the same error, but this works:

using PyCall
signal = pyimport("signal")
signal["signal"](signal["SIGINT"], signal["SIG_IGN"])

Thanks for your help.

stevengj commented 7 years ago

Re-opening since this seems like it is something that should be fixed somewhere by default. I'm also not sure what we should be doing here, since Julia also installs its own signal handlers and I'm not sure how they are interacting with the Python ones.

What did original_sigint = signal["getsignal"](signal["SIGINT"]) return? In Python 2.7, I got PyObject <built-in function default_int_handler>

bluesmoon commented 7 years ago

I'm using python 3.4.3:

julia> using PyCall
julia> PyCall.pyversion
v"3.4.3"
julia> @pyimport snowflake.connector as snowflake
julia> signal = pyimport("signal")
PyObject <module 'signal' (built-in)>
julia> original_sigint = signal["getsignal"](signal["SIGINT"])
julia> show(original_sigint)
nothing

>>> import signal
>>> import snowflake.connector as snowflake
>>> original_sigint = signal.getsignal(signal.SIGINT)
>>> print(original_sigint)
<built-in function default_int_handler>
>>> type(original_sigint)
<class 'builtin_function_or_method'>

stevengj commented 7 years ago

The situation is a bit odd because we really don't want libpython to install its own SIGINT handler, since Julia has already installed its own. (On the other hand, the "signal handlers" installed by Python's signal.signal API aren't really the low-level system signal-handler functions.) I probably have to look at the CPython source code to figure out whether there is something appropriate we can do here.

JuliaPy / PyCall.jl