search

python / cpython

The Python programming language
https://www.python.org
Other
63.39k stars 30.36k forks source link

Use ctypes to load C library in the win11 system will crash #105955

Closed stgfeng closed 1 year ago

stgfeng commented 1 year ago

Bug report

I used ctypes to load C library in the win11 system, python will crash when the interface is called. But it can be used normally in the win10 system.

Code like this

from ctypes import *
lib = CDLL("./SR7Link.dll")
class SR7IF_ETHERNET_CONFIG(Structure):
    _fields_ = [('abyIpAddr', c_ubyte * 4)]
ip = SR7IF_ETHERNET_CONFIG()
ip.abyIpAddr = (192,168,0,10)
lib.SR7IF_EthernetOpen(0, ip.abyIpAddr)

Your environment

Windows 11 hone 22H2 22621.1702

python 3.11.4 amd64

The C library

SR7Link.zip

sunmy2019 commented 1 year ago

Why are you assuming it can also be used on Windows 11?

What is inside your dll? Letting others open your dll is not safe for others.

You need to provide strong evidence that this is safe.

eryksun commented 1 year ago

Please provide the exact definition of the SR7IF_ETHERNET_CONFIG structure and SR7IF_EthernetOpen prototype from the C API header. I searched but couldn't find it. I assume it's in a development kit that can be downloaded freely or licensed.

stgfeng commented 1 year ago

Please provide the exact definition of the SR7IF_ETHERNET_CONFIG structure and SR7IF_EthernetOpen prototype from the C API header. I searched but couldn't find it. I assume it's in a development kit that can be downloaded freely or licensed.

Yes, it's a development kit. It can be downloaded freely. The new zip file contain the C API header. SR7Link.zip

stgfeng commented 1 year ago

Why are you assuming it can also be used on Windows 11?

What is inside your dll? Letting others open your dll is not safe for others.

You need to provide strong evidence that this is safe.

This is our company's development kit. You can download the SDK from https://www.cnsszn.com/download.html.

I'm sure it can be used on Windows 11, the SDK's VS demo or QT demo can be used normally in the win11 system.

sunmy2019 commented 1 year ago

Use ctypes to load C library in the win11 system will crash

Can you reproduce this without using your company's development kit?

How can you tell it is a bug of ctypes? What if it's something related to the Windows library? What if it's something is incompatible with your development kit?

Also, we cannot get the source code of your SDK. It is impractical for us to debug your SDK.

sunmy2019 commented 1 year ago

If you want to let us debug ctypes, my advice would be to provide a reproducible example that does not involve your SDK.

Otherwise, there is not much that we can do.

eryksun commented 1 year ago

For everyone's reference, here are the relevant declarations and definitions from the C header "SR7Link.h":

#ifdef WIN32
#define  SR7_IF_API __declspec(dllexport)
#else
#define  SR7_IF_API extern
#endif

typedef struct {
    unsigned char   abyIpAddress[4];
} SR7IF_ETHERNET_CONFIG;

SR7_IF_API int SR7IF_EthernetOpen(unsigned int lDeviceId, SR7IF_ETHERNET_CONFIG* pEthernetConfig);

Here's how I would translate this to Python:

import ctypes

lib = ctypes.CDLL('./SR7Link.dll')

class SR7IF_ETHERNET_CONFIG(ctypes.Structure):
    __slots__ = ()
    _fields_ = (
       ('abyIpAddress', ctypes.c_ubyte * 4),
    )

lib.SR7IF_EthernetOpen.restype = ctypes.c_int
lib.SR7IF_EthernetOpen.argtypes = (
    ctypes.c_uint, # lDeviceId
    ctypes.POINTER(SR7IF_ETHERNET_CONFIG), # pEthernetConfig
)

ip = SR7IF_ETHERNET_CONFIG(abyIpAddress=(192, 168, 0, 10))
result = lib.SR7IF_EthernetOpen(0, ctypes.byref(ip))

That said, I don't see a serious error in your initial example. It lacks the prototype for SR7IF_EthernetOpen, but the default conversion of 0 works fine for an unsigned int parameter, and passing the ctypes array ip.abyIpAddr isn't effectively different from passing ctypes.byref(ip) in this case. They're both passed as a pointer to the same 4 bytes in memory.

If you can't provide more information on the nature of the crash that demonstrates that the Python runtime, ctypes, or libffi is at fault, then there's nothing more that I can do here. For example, provide the Windows event log for the crash that includes the exception code and faulting module (e.g. "python311.dll", "_ctypes.pyd", "libffi-8.dll"). Even better would be to run Python under a native debugger (e.g. windbg, cdb) -- or configure one for automatic, postmortem debugging -- and provide the output from the debugger command !analyze -v or a minidump created by the debugger command .dump /m <filename>.

stgfeng commented 1 year ago

For everyone's reference, here are the relevant declarations and definitions from the C header "SR7Link.h":

#ifdef WIN32
#define  SR7_IF_API __declspec(dllexport)
#else
#define  SR7_IF_API extern
#endif

typedef struct {
    unsigned char abyIpAddress[4];
} SR7IF_ETHERNET_CONFIG;

SR7_IF_API int SR7IF_EthernetOpen(unsigned int lDeviceId, SR7IF_ETHERNET_CONFIG* pEthernetConfig);

Here's how I would translate this to Python:

import ctypes

lib = ctypes.CDLL('./SR7Link.dll')

class SR7IF_ETHERNET_CONFIG(ctypes.Structure):
    __slots__ = ()
    _fields_ = (
       ('abyIpAddress', ctypes.c_ubyte * 4),
    )

lib.SR7IF_EthernetOpen.restype = ctypes.c_int
lib.SR7IF_EthernetOpen.argtypes = (
    ctypes.c_uint, # lDeviceId
    ctypes.POINTER(SR7IF_ETHERNET_CONFIG), # pEthernetConfig
)

ip = SR7IF_ETHERNET_CONFIG(abyIpAddress=(192, 168, 0, 10))
result = lib.SR7IF_EthernetOpen(0, ctypes.byref(ip))

That said, I don't see a serious error in your initial example. It lacks the prototype for SR7IF_EthernetOpen, but the default conversion of 0 works fine for an unsigned int parameter, and passing the ctypes array ip.abyIpAddr isn't effectively different from passing ctypes.byref(ip) in this case. They're both passed as a pointer to the same 4 bytes in memory.

If you can't provide more information on the nature of the crash that demonstrates that the Python runtime, ctypes, or libffi is at fault, then there's nothing more that I can do here. For example, provide the Windows event log for the crash that includes the exception code and faulting module (e.g. "python311.dll", "_ctypes.pyd", "libffi-8.dll"). Even better would be to run Python under a native debugger (e.g. windbg, cdb) -- or configure one for automatic, postmortem debugging -- and provide the output from the debugger command !analyze -v or a minidump created by the debugger command .dump /m <filename>.

The contents of test.py are

import ctypes

lib = ctypes.CDLL('./SR7Link.dll')

class SR7IF_ETHERNET_CONFIG(ctypes.Structure):
    __slots__ = ()
    _fields_ = (
       ('abyIpAddress', ctypes.c_ubyte * 4),
    )

lib.SR7IF_EthernetOpen.restype = ctypes.c_int
lib.SR7IF_EthernetOpen.argtypes = (
    ctypes.c_uint, # lDeviceId
    ctypes.POINTER(SR7IF_ETHERNET_CONFIG), # pEthernetConfig
)

ip = SR7IF_ETHERNET_CONFIG(abyIpAddress=(192, 168, 0, 10))
result = lib.SR7IF_EthernetOpen(0, ctypes.byref(ip))

The output of windbg

NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\atlmfc.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\ObjectiveC.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\concurrency.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\cpp_rest.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\stl.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Data.Json.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Devices.Geolocation.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Devices.Sensors.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Media.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\windows.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\winrt.natvis' CommandLine: "C:\Program Files\Python311\python.exe" C:\test\SR7LinkDemo\Python\64bit\test.py Starting directory: C:\test\SR7LinkDemo\Python\64bit

* Path validation summary ** Response Time (ms) Location Deferred srv Symbol search path is: srv Executable search path is: ModLoad: 00007ff6140f0000 00007ff61410a000 python.exe ModLoad: 00007ffbe9690000 00007ffbe98a4000 ntdll.dll ModLoad: 00007ffbe89e0000 00007ffbe8aa2000 C:\Windows\System32\KERNEL32.DLL ModLoad: 00007ffbe6e00000 00007ffbe71a3000 C:\Windows\System32\KERNELBASE.dll ModLoad: 00007ffbe7370000 00007ffbe7481000 C:\Windows\System32\ucrtbase.dll ModLoad: 00007ffb7d920000 00007ffb7deed000 C:\Program Files\Python311\python311.dll ModLoad: 00007ffbe8960000 00007ffbe89d1000 C:\Windows\System32\WS2_32.dll ModLoad: 00007ffbdd550000 00007ffbdd56b000 C:\Program Files\Python311\VCRUNTIME140.dll ModLoad: 00007ffbd15f0000 00007ffbd15fa000 C:\Windows\SYSTEM32\VERSION.dll ModLoad: 00007ffbe7c10000 00007ffbe7cb7000 C:\Windows\System32\msvcrt.dll ModLoad: 00007ffbe87b0000 00007ffbe88c7000 C:\Windows\System32\RPCRT4.dll ModLoad: 00007ffbe7b60000 00007ffbe7c0e000 C:\Windows\System32\ADVAPI32.dll ModLoad: 00007ffbe7dc0000 00007ffbe7e64000 C:\Windows\System32\sechost.dll ModLoad: 00007ffbe6390000 00007ffbe63b8000 C:\Windows\SYSTEM32\bcrypt.dll (ab4.1d20): Break instruction exception - code 80000003 (first chance) ntdll!LdrpDoDebuggerBreak+0x30: 00007ffb`e976cea4 cc int 3 0:000> !analyze -v /*** / / Exception Analysis / /***

KEY_VALUES_STRING: 1

Key  : Analysis.CPU.mSec
Value: 2234

Key  : Analysis.DebugAnalysisManager
Value: Create

Key  : Analysis.Elapsed.mSec
Value: 213008

Key  : Analysis.Init.CPU.mSec
Value: 8093

Key  : Analysis.Init.Elapsed.mSec
Value: 2016159

Key  : Analysis.Memory.CommitPeak.Mb
Value: 100

Key  : Timeline.OS.Boot.DeltaSec
Value: 2648

Key  : Timeline.Process.Start.DeltaSec
Value: 1369

Key  : WER.OS.Branch
Value: ni_release

Key  : WER.OS.Timestamp
Value: 2022-05-06T12:50:00Z

Key  : WER.OS.Version
Value: 10.0.22621.1

Key  : WER.Process.Version
Value: 3.11.4150.1013

NTGLOBALFLAG: 70

PROCESS_BAM_CURRENT_THROTTLED: 0

PROCESS_BAM_PREVIOUS_THROTTLED: 0

APPLICATION_VERIFIER_FLAGS: 0

EXCEPTION_RECORD: (.exr -1) ExceptionAddress: 00007ffbe976cea4 (ntdll!LdrpDoDebuggerBreak+0x0000000000000030) ExceptionCode: 80000003 (Break instruction exception) ExceptionFlags: 00000000 NumberParameters: 1 Parameter[0]: 0000000000000000

FAULTING_THREAD: 00001d20

PROCESS_NAME: python.exe

ERROR_CODE: (NTSTATUS) 0x80000003 - { }

EXCEPTION_CODE_STR: 80000003

EXCEPTION_PARAMETER1: 0000000000000000

STACK_TEXT:
00000041107eee30 00007ffbe97661b9 : 00007ffbe97dcc00 0000004110480000 0000004110480000 0000004110480000 : ntdll!LdrpDoDebuggerBreak+0x30 00000041107eee70 00007ffbe9759a00 : 0000000000000000 0000000000000000 0000000000000000 0000000000000001 : ntdll!LdrpInitializeProcess+0x1cf9 00000041107ef240 00007ffbe9703e53 : 00000041107ef590 00007ffbe9690000 0000004110480050 00000041104827ee : ntdll!_LdrpInitialize+0x55b74 00000041107ef2c0 00007ffbe9703d7e : 00000041107ef590 0000000000000000 00000041107ef590 0000000000000000 : ntdll!LdrpInitializeInternal+0x6b 00000041107ef540 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : ntdll!LdrInitializeThunk+0xe

SYMBOL_NAME: ntdll!LdrpDoDebuggerBreak+30

MODULE_NAME: ntdll

IMAGE_NAME: ntdll.dll

STACK_COMMAND: dt ntdll!LdrpLastDllInitializer BaseDllName ; dt ntdll!LdrpFailureData ; ~0s ; .cxr ; kb

FAILURE_BUCKET_ID: BREAKPOINT_80000003_ntdll.dll!LdrpDoDebuggerBreak

OS_VERSION: 10.0.22621.1

BUILDLAB_STR: ni_release

OSPLATFORM_TYPE: x64

OSNAME: Windows 10

IMAGE_VERSION: 10.0.22621.1485

FAILURE_ID_HASH: {06f54d4d-201f-7f5c-0224-0b1f2e1e15a5}

Followup: MachineOwner

0:000> ExceptionAddress: 00007ffbe976cea4 (ntdll!LdrpDoDebuggerBreak+0x0000000000000030) ExceptionCode: 80000003 (Break instruction exception) ExceptionFlags: 00000000 NumberParameters: 1 Parameter[0]: 0000000000000000 0:000> ExceptionAddress: 00007ffbe976cea4 (ntdll!LdrpDoDebuggerBreak+0x0000000000000030) ExceptionCode: 80000003 (Break instruction exception) ExceptionFlags: 00000000 NumberParameters: 1 Parameter[0]: 0000000000000000 0:000> ExceptionAddress: 00007ffbe976cea4 (ntdll!LdrpDoDebuggerBreak+0x0000000000000030) ExceptionCode: 80000003 (Break instruction exception) ExceptionFlags: 00000000 NumberParameters: 1 Parameter[0]: 0000000000000000 0:000> ExceptionAddress: 00007ffbe976cea4 (ntdll!LdrpDoDebuggerBreak+0x0000000000000030) ExceptionCode: 80000003 (Break instruction exception) ExceptionFlags: 00000000 NumberParameters: 1 Parameter[0]: 0000000000000000

sunmy2019 commented 1 year ago

Isn't this debug report from Win 10?

stgfeng commented 1 year ago

Test PC information

CPU 12th Gen Intel(R) Core(TM) i7-12700H 2.30 GHz RAM 16.0 GB SYSTEM Windows 11 HOME VERSION 22H2 Python 3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)] on win32 WinDbg:10.0.22621.1778 AMD64

sunmy2019 commented 1 year ago

STACK_TEXT: 00000041107eee30 00007ffbe97661b9 : 00007ffbe97dcc00 0000004110480000 0000004110480000 0000004110480000 : ntdll!LdrpDoDebuggerBreak+0x30 00000041107eee70 00007ffbe9759a00 : 0000000000000000 0000000000000000 0000000000000000 0000000000000001 : ntdll!LdrpInitializeProcess+0x1cf9 00000041107ef240 00007ffbe9703e53 : 00000041107ef590 00007ffbe9690000 0000004110480050 00000041104827ee : ntdll!_LdrpInitialize+0x55b74 00000041107ef2c0 00007ffbe9703d7e : 00000041107ef590 0000000000000000 00000041107ef590 0000000000000000 : ntdll!LdrpInitializeInternal+0x6b 00000041107ef540 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : ntdll!LdrInitializeThunk+0xe

This stack does not look related to CPython.

AA-Turner commented 1 year ago

Closing, no response from reporter.

A