search

ansible / pylibssh

Python bindings specific to Ansible use case for libssh https://www.libssh.org/
https://ansible-pylibssh.rtfd.io
GNU Lesser General Public License v2.1
59 stars 30 forks source link

SIGSEGV when running ssh_channel.exec_command() over different channels #657

Open kucharskim opened 2 days ago

kucharskim commented 2 days ago
SUMMARY

On OpenBSD -current as of 2024-11-06 I have following backtrace from a core dump:

$ egdb -quiet -batch -x commands.txt -e /usr/local/bin/python3.11 -c python3.11.core                                                                                                                                            
[New process 517943]
Core was generated by `python3.11'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00000f3db85e2818 in __pyx_type_11pylibsshext_7session_Session () from /usr/local/lib/python3.11/site-packages/pylibsshext/session.cpython-311.so
+show version
GNU gdb (GDB) 13.2
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-unknown-openbsd7.6".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
+thread apply all bt

Thread 1 (process 517943):
+bt
#0  0x00000f3db85e2818 in __pyx_type_11pylibsshext_7session_Session () from /usr/local/lib/python3.11/site-packages/pylibsshext/session.cpython-311.so
#1  0x00000f3d570a76c4 in ssh_packet_socket_controlflow_callback (code=<optimized out>, userdata=0xf3e0f34c800) at /usr/obj/ports/libssh-0.10.6/libssh-0.10.6/src/packet.c:1443
#2  0x00000f3d570b342b in ssh_socket_pollcallback (p=0xf3d74e617e0, fd=3, revents=<optimized out>, v_s=0xf3e0f315230) at /usr/obj/ports/libssh-0.10.6/libssh-0.10.6/src/socket.c:386
#3  0x00000f3d570af365 in ssh_poll_ctx_dopoll (ctx=0xf3d74e5a8d0, timeout=<optimized out>) at /usr/obj/ports/libssh-0.10.6/libssh-0.10.6/src/poll.c:743
#4  0x00000f3d570b0bb6 in ssh_handle_packets (session=0xf3e0f34c800, timeout=<optimized out>) at /usr/obj/ports/libssh-0.10.6/libssh-0.10.6/src/session.c:686
#5  0x00000f3d570b095b in ssh_handle_packets_termination (session=0xf3e0f34c800, timeout=<optimized out>, fct=0xf3d57091560 <ssh_channel_open_termination>, user=0xf3d74e45e70) at /usr/obj/ports/libssh-0.10.6/libssh-0.10.6/src/session.c:755
#6  0x00000f3d5708e195 in channel_open (channel=0xf3d74e45e70, type=0xf3d57070c84 "session", window=64000, maxpacket=32768, payload=0x0) at /usr/obj/ports/libssh-0.10.6/libssh-0.10.6/src/channels.c:364
#7  0x00000f3df5faafb9 in __pyx_pf_11pylibsshext_7channel_7Channel___cinit__ (__pyx_v_self=0xf3d7c974cc0, __pyx_v_session=<error reading variable: Cannot access memory at address 0x0>) at /tmp/.tmp-ansible-pylibssh-pep517-6bgc_omf/src/src/pylibsshext/channel.c:4041
#8  __pyx_pw_11pylibsshext_7channel_7Channel_1__cinit__ (__pyx_v_self=0xf3d7c974cc0, __pyx_args=<optimized out>, __pyx_kwds=<optimized out>) at /tmp/.tmp-ansible-pylibssh-pep517-6bgc_omf/src/src/pylibsshext/channel.c:3939
#9  __pyx_tp_new_11pylibsshext_7channel_Channel (t=<optimized out>, a=<optimized out>, k=<optimized out>) at /tmp/.tmp-ansible-pylibssh-pep517-6bgc_omf/src/src/pylibsshext/channel.c:8577
#10 0x00000f3e06729af9 in type_call (type=0xf3df5fb86a8 <__pyx_type_11pylibsshext_7channel_Channel>, args=0xf3d7c927d90, kwds=0x0) at Objects/typeobject.c:1091
#11 0x00000f3e066a4d68 in _PyObject_MakeTpCall (tstate=0xf3e06a019e0 <_PyRuntime+166184>, callable=0xf3df5fb86a8 <__pyx_type_11pylibsshext_7channel_Channel>, args=0x72963ec892a8, nargs=<optimized out>, keywords=0x0) at Objects/call.c:214
#12 0x00000f3db85d4719 in __pyx_pf_11pylibsshext_7session_7Session_30new_channel (__pyx_v_self=0xf3d7c95f240) at /tmp/.tmp-ansible-pylibssh-pep517-6bgc_omf/src/src/pylibsshext/session.c:11379
#13 __pyx_pw_11pylibsshext_7session_7Session_31new_channel (__pyx_v_self=0xf3d7c95f240, __pyx_args=<optimized out>, __pyx_nargs=<optimized out>, __pyx_kwds=<optimized out>) at /tmp/.tmp-ansible-pylibssh-pep517-6bgc_omf/src/src/pylibsshext/session.c:11330
#14 0x00000f3e066a564d in _PyObject_VectorcallTstate (tstate=0xf3e06a019e0 <_PyRuntime+166184>, callable=0xf3dc50cd490, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at ./Include/internal/pycore_call.h:92
#15 PyObject_Vectorcall (callable=0xf3dc50cd490, args=0xf3e0f31acb0, nargsf=2097152, kwnames=0xf3db85e2818 <__pyx_type_11pylibsshext_7session_Session>) at Objects/call.c:299
#16 0x00000f3e067be1cd in _PyEval_EvalFrameDefault (tstate=0xf3e06a019e0 <_PyRuntime+166184>, frame=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:7301
#17 0x00000f3e067adb1e in _PyEval_EvalFrame (tstate=0xf3e06a019e0 <_PyRuntime+166184>, frame=0xf3d7b85b020, throwflag=<error reading variable: Cannot access memory at address 0x0>) at ./Include/internal/pycore_ceval.h:73
#18 _PyEval_Vector (tstate=0xf3e06a019e0 <_PyRuntime+166184>, func=0xf3de6329f80, locals=<optimized out>, args=<error reading variable: Cannot access memory at address 0x0>, argcount=<error reading variable: Cannot access memory at address 0x0>, kwnames=<error reading variable: Cannot access memory at address 0x0>) at Python/ceval.c:6434
#19 PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at Python/ceval.c:1148
#20 0x00000f3e0681e6e9 in run_eval_code_obj (tstate=0xf3e06a019e0 <_PyRuntime+166184>, co=0xf3dfd2eb000, globals=0xf3de6350400, locals=0xf3de6350400) at Python/pythonrun.c:1741
#21 run_mod (mod=<optimized out>, filename=<optimized out>, globals=0xf3de6350400, locals=0xf3de6350400, flags=<optimized out>, arena=<optimized out>) at Python/pythonrun.c:1762
#22 0x00000f3e0681e1d5 in pyrun_file (fp=0xf3d26a66dd0 <usual>, filename=0xf3d7c91d990, start=<error reading variable: Cannot access memory at address 0x101>, globals=0xf3de6350400, locals=0xf3de6350400, closeit=1, flags=0x72963ec89710) at Python/pythonrun.c:1657
#23 _PyRun_SimpleFileObject (fp=0xf3d26a66dd0 <usual>, filename=0xf3d7c91d990, closeit=1, flags=0x72963ec89710) at Python/pythonrun.c:440
#24 0x00000f3e0681d3b3 in _PyRun_AnyFileObject (fp=0xf3d26a66dd0 <usual>, filename=0xf3d7c91d990, closeit=1, flags=0x72963ec89710) at Python/pythonrun.c:79
#25 0x00000f3e06847172 in pymain_run_file_obj (program_name=0xf3de63505f0, filename=0xf3d7c91d990, skip_source_first_line=0) at Modules/main.c:360
#26 pymain_run_file (config=<optimized out>) at Modules/main.c:379
#27 pymain_run_python (exitcode=<optimized out>) at Modules/main.c:605
#28 Py_RunMain () at Modules/main.c:684
#29 0x00000f3e06848095 in pymain_main (args=0x72963ec89a28) at Modules/main.c:714
#30 0x00000f3e068484bc in Py_BytesMain (argc=<optimized out>, argv=0xf3e0f31acb0) at Modules/main.c:738
#31 0x00000f3b1df6994b in ?? ()
#32 0x00000f3b1df69820 in ?? ()
#33 0x0000000000000000 in ?? ()
+info threads
  Id   Target Id         Frame 
* 1    process 517943    0x00000f3db85e2818 in __pyx_type_11pylibsshext_7session_Session () from /usr/local/lib/python3.11/site-packages/pylibsshext/session.cpython-311.so
+info locals
No symbol table info available.
ISSUE TYPE
PYLISSH and LIBSSH VERSION
$ python3 version.py
__full_version__='<pylibsshext v1.2.2 with libssh v0.10.6>'
__libssh_version__='0.10.6'
__version__='1.2.2'
__version_info__=(1, 2, 2)
OS / ENVIRONMENT

This is similar bug report to https://github.com/ansible/pylibssh/issues/645

OpenBSD 7.6-current (GENERIC.MP) #417: Wed Nov  6 15:53:38 MST 2024
    deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
$ pkg_info -qI python libssh py3-ansible-libssh
python-3.11.10p1
libssh-0.10.6p0
py3-ansible-libssh-1.2.2
STEPS TO REPRODUCE
$ cat test2.py                                                                                                                                                                                                                           
#!/usr/bin/env python3

from pylibsshext.errors import LibsshSessionException
from pylibsshext.session import Session

import logging

HOST = "examplemachine1"
USER = "root"
TIMEOUT = 30
PORT = 22

ssh = Session()

#ssh.set_log_level(logging.DEBUG)

try:
    ssh.connect(
        host=HOST,
        user=USER,
        timeout=TIMEOUT,
        port=PORT,
        # proxycommand="ssh -q -W %h:%p ks2",
    )
except LibsshSessionException as ex:
    print(f"Failed to connect to {HOST}:{PORT} over SSH: {ex!s}")

print(f"ssh.is_connected={ssh.is_connected}")

def run_cmd(cmd):
    print(f"Running {cmd}...", flush=True)
    print(f"ssh.is_connected={ssh.is_connected}")
    ssh_channel = ssh.new_channel()
    print(f"ssh_channel={ssh_channel}", flush=True)

    print(f"Executing exec_command()...", flush=True)
    try:
        cmd_resp = ssh_channel.exec_command(cmd)
        print(f"Executing exec_command()... done", flush=True)

        print(f"stdout type: {type(cmd_resp.stdout)}")
        print(f"stdout:\n{cmd_resp.stdout.decode()}\n")
        print(f"stderr type: {type(cmd_resp.stderr)}")
        print(f"stderr:\n{cmd_resp.stderr.decode()}\n")
        print(f"return code: {cmd_resp.returncode}")
    finally:
        print(f"Executing ssh_channel.close()...", flush=True)
        ssh_channel.close()

run_cmd("uptime")
run_cmd("ls")
run_cmd("hostname")

print("Closing connection...", flush=True)
ssh.close()
EXPECTED RESULTS

Execution of below script should work all the time, but it core dumps intermittently.

$ python3 test2.py
ACTUAL RESULTS
$ python3 test2.py
ssh.is_connected=1
Running uptime...
ssh.is_connected=1
ssh_channel=<pylibsshext.channel.Channel object at 0x73606b60b40>
Executing exec_command()...
Executing exec_command()... done
stdout type: <class 'bytes'>
stdout:
 6:17PM  up  4:58, 0 users, load averages: 0.01, 0.02, 0.00

stderr type: <class 'bytes'>
stderr:

return code: 0
Executing ssh_channel.close()...
Running ls...
ssh.is_connected=1
Segmentation fault (core dumped)
kucharskim commented 2 days ago

Execution log with ssh's LogLevel DEBUG3 and Phython's ssh.set_log_level(logging.DEBUG) test2-output-ansible-libssh-1.2.2-v001.txt

kucharskim commented 2 days ago

This issue was triggered by similar problem from https://github.com/ansible/pylibssh/issues/645