IronLanguages / ironpython3

Implementation of Python 3.x for .NET Framework that is built on top of the Dynamic Language Runtime.
Apache License 2.0
2.47k stars 286 forks source link

`ctypes` does not work on Windows and has memory corruption #1526

Open strongly-typed opened 2 years ago

strongly-typed commented 2 years ago

I tried to use ctypes to interface to some C/C++ code as DLL with IronPython on Windows. I got random segfaults and errors in malloc. I am trying to boil it down and moved to a simple project that works with CPython but fails with IronPython2.7 and IronPython3.4-beta.

The project to test with is Python-using-C-CPP-libraries-with-ctypes. Great shout-out to @sol-prog

I am using Windows 10 21H1 OS build 19044.1865 and Visual Studio Community 2022. Detailed environment information at the bottom.

I used the released IronPython versions and also built IronPython3.4 from source (so trying that the same compiler is used for the DLL and IronPython binaries).

When I compile the DLL using MSVC (cbmp.dll) and run it with IronPython3.4 the code does not run due to some internal memory corruption.

PS C:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\using_cpp_libs_from_python> ..\..\ironpython3\bin\Debug\net6.0\ipy.exe .\test_cbmp.py
Error in function: BMP_fill_region at line: 114
The region does not fit in the image!
PS C:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\using_cpp_libs_from_python> ..\..\ironpython3\bin\Debug\net46\ipy.exe .\test_cbmp.py
Error in function: BMP_fill_region at line: 114
The region does not fit in the image!

When running with Python3 the code works just fine and produces valid output.

PS C:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\using_cpp_libs_from_python> python3 .\test_cbmp.py
500 500 3

In cbmp.py I had to modify the loading of the library to:

# Find the library and load it
try:
    cbmp = ctypes.cdll.LoadLibrary(".\cbmp")
except OSError:
    print("Unable to load the system C library")
    sys.exit()

I compiled the DLL in the following way (from the "x64 Native Tools Command Prompt for VS 2022")

c:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\using_cpp_libs_from_python>cl /LD /std:c++17 /EHsc /W3 /DBUILD_CBMP cbmp.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 19.32.31332 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

cbmp.cpp
Microsoft (R) Incremental Linker Version 14.32.31332.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:cbmp.dll
/dll
/implib:cbmp.lib
cbmp.obj
   Creating library cbmp.lib and object cbmp.exp

x86 (32-bit) Test

I also build IronPython x86 and the DLL with x86 to test:

From "x86 Native Tools Command Prompt for VS 2022":

c:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\using_cpp_libs_from_python>cl /LD /std:c++17 /EHsc /W3 /DBUILD_CBMP cbmp.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 19.32.31332 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cbmp.cpp
Microsoft (R) Incremental Linker Version 14.32.31332.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:cbmp.dll
/dll
/implib:cbmp.lib
cbmp.obj
   Creating library cbmp.lib and object cbmp.exp

Build Python for x86. Only the net46 version seemed to be build:

PS C:\Users\Sascha\Desktop\working_directory\ironpython3> .\make.ps1 -platform x86 debug

This gives some access violation. The code works with CPython and the DLL seems to be fine. I guess something in ctypes is broken in IronPython?!

Or what are the best/correct compiler parameters I should use to compile the DLL with? To match calling conventions, pointer sizes, et cetera?

PS C:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\using_cpp_libs_from_python> ..\..\ironpython3\bin\Debug\net46\ipy32.exe .\test_cbmp.py

Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at InteropInvoker(IntPtr , Object , Object , Object , Object , Object , Object , Object , Object , Object , Object[] )
   at CallSite.Target(Closure , CallSite , Object , Object , Object , Object , Object , Object , Object , Object , Object , Object )
   at System.Dynamic.UpdateDelegates.UpdateAndExecute10[T0,T1,T2,T3,T4,T5,T6,T7,T8,T9,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2, T3 arg3, T4 arg4, T5 arg5, T6 arg6, T7 arg7, T8 arg8, T9 arg9)
   at Microsoft.Scripting.Interpreter.DynamicInstruction`11.Run(InterpretedFrame frame) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Interpreter\Instructions\DynamicInstructions.Generated.cs:line 352
   at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Interpreter\Interpreter.cs:line 91
   at Microsoft.Scripting.Interpreter.LightLambda.Run12[T0,T1,T2,T3,T4,T5,T6,T7,T8,T9,T10,T11,TRet](T0 arg0, T1 arg1, T2 arg2, T3 arg3, T4 arg4, T5 arg5, T6 arg6, T7 arg7, T8 arg8, T9 arg9, T10 arg10, T11 arg11) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Interpreter\LightLambda.Generated.cs:line 506
   at CallSite.Target(Closure , CallSite , CodeContext , Object , Object , Object , Object , Object , Object , Object , Object , Object , Object )
   at Microsoft.Scripting.Interpreter.DynamicInstruction`12.Run(InterpretedFrame frame) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Interpreter\Instructions\DynamicInstructions.Generated.cs:line 377
   at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Interpreter\Interpreter.cs:line 91
   at Microsoft.Scripting.Interpreter.LightLambda.Run1[T0,TRet](T0 arg0) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Interpreter\LightLambda.Generated.cs:line 55
   at IronPython.Compiler.RuntimeScriptCode.InvokeTarget(Scope scope) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\IronPython\Compiler\RuntimeScriptCode.cs:line 64
   at IronPython.Compiler.RuntimeScriptCode.Run(Scope scope) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\IronPython\Compiler\RuntimeScriptCode.cs:line 44
   at IronPython.Hosting.PythonCommandLine.RunFileWorker(String fileName) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\IronPython\Hosting\PythonCommandLine.cs:line 603
   at IronPython.Hosting.PythonCommandLine.RunFile(String fileName) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\IronPython\Hosting\PythonCommandLine.cs:line 558
   at Microsoft.Scripting.Hosting.Shell.CommandLine.Run() in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Hosting\Shell\CommandLine.cs:line 132
   at IronPython.Hosting.PythonCommandLine.Run() in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\IronPython\Hosting\PythonCommandLine.cs:line 113
   at Microsoft.Scripting.Hosting.Shell.CommandLine.Run(ScriptEngine engine, IConsole console, ConsoleOptions options) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Hosting\Shell\CommandLine.cs:line 100
   at Microsoft.Scripting.Hosting.Shell.ConsoleHost.RunCommandLine() in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs:line 384
   at Microsoft.Scripting.Hosting.Shell.ConsoleHost.ExecuteInternal() in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs:line 319
   at PythonConsoleHost.ExecuteInternal() in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\IronPythonConsole\Console.cs:line 165
   at Microsoft.Scripting.Hosting.Shell.ConsoleHost.Execute() in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs:line 297
   at Microsoft.Scripting.Hosting.Shell.ConsoleHost.Run(String[] args) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\DLR\Src\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs:line 197
   at PythonConsoleHost.Main(String[] args) in C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\IronPythonConsole\Console.cs:line 199

Environment

PS C:\Users\Sascha\Desktop\working_directory\ironpython3> .\make.ps1 debug

Welcome to .NET 6.0!
---------------------
SDK Version: 6.0.302
PS C:\Users\Sascha\Desktop\working_directory\ironpython3> dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.302
 Commit:    c857713418

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19044
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\6.0.302\

global.json file:
  Not found

Host:
  Version:      6.0.7
  Architecture: x64
  Commit:       0ec02c8c96

.NET SDKs installed:
  6.0.302 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
PS C:\Users\Sascha\Desktop\working_directory\ironpython3> .\bin\Debug\net6.0\ipy.exe
IronPython 3.4.0b1 DEBUG (3.4.0.0010)
[.NETCoreApp,Version=v6.0 on .NET 6.0.7 (64-bit)] on win32
Type "help", "copyright", "credits" or "license" for more information.
PS C:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\using_cpp_libs_from_python> ..\..\ironpython3\bin\Debug\net6.0\ipy.exe -m sysconfig
Platform: "win32"
Python version: "3.4"
Current installation scheme: "nt"

Paths:
        data = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0"
        include = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0\Include"
        platinclude = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0\Include"
        platlib = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0\Lib\site-packages"
        platstdlib = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0\Lib"
        purelib = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0\Lib\site-packages"
        scripts = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0\Scripts"
        stdlib = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0\Lib"

Variables:
        BINDIR = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0"
        BINLIBDEST = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0\Lib"
        EXE = ".exe"
        EXT_SUFFIX = ".pyd"
        INCLUDEPY = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0\Include"
        LIBDEST = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0\Lib"
        SO = ".pyd"
        VERSION = "34"
        abiflags = ""
        base = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0"
        exec_prefix = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0"
        implementation = "IronPython"
        implementation_lower = "ironpython"
        installed_base = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0"
        installed_platbase = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0"
        platbase = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0"
        prefix = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0"
        projectbase = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0"
        py_version = "3.4.0b1"
        py_version_nodot = "34"
        py_version_short = "3.4"
        srcdir = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net6.0"
        userbase = "C:\Users\Sascha\AppData\Roaming\Python"
PS C:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\using_cpp_libs_from_python> ..\..\ironpython3\bin\Debug\net46\ipy.exe
IronPython 3.4.0b1 DEBUG (3.4.0.0010)
[.NETFramework,Version=v4.6 on .NET Framework 4.8.4515.0 (64-bit)] on win32
PS C:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\using_cpp_libs_from_python> ..\..\ironpython3\bin\Debug\net46\ipy.exe -m sysconfig
Platform: "win32"
Python version: "3.4"
Current installation scheme: "nt"

Paths:
        data = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46"
        include = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46\Include"
        platinclude = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46\Include"
        platlib = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46\Lib\site-packages"
        platstdlib = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46\Lib"
        purelib = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46\Lib\site-packages"
        scripts = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46\Scripts"
        stdlib = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46\Lib"

Variables:
        BINDIR = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46"
        BINLIBDEST = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46\Lib"
        EXE = ".exe"
        EXT_SUFFIX = ".pyd"
        INCLUDEPY = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46\Include"
        LIBDEST = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46\Lib"
        SO = ".pyd"
        VERSION = "34"
        abiflags = ""
        base = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46"
        exec_prefix = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46"
        implementation = "IronPython"
        implementation_lower = "ironpython"
        installed_base = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46"
        installed_platbase = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46"
        platbase = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46"
        prefix = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46"
        projectbase = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46"
        py_version = "3.4.0b1"
        py_version_nodot = "34"
        py_version_short = "3.4"
        srcdir = "C:\Users\Sascha\Desktop\working_directory\ironpython3\bin\Debug\net46"
        userbase = "C:\Users\Sascha\AppData\Roaming\Python"

"x64 Native Tools Command Prompt for VS 2022"

c:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\using_cpp_libs_from_python>cl.exe
Microsoft (R) C/C++ Optimizing Compiler Version 19.32.31332 for x64
PS C:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\using_cpp_libs_from_python> python3 -m sysconfig
Platform: "win-amd64"
Python version: "3.10"
Current installation scheme: "nt"

Paths:
        data = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0"
        include = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\Include"
        platinclude = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\Include"
        platlib = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\Lib\site-packages"
        platstdlib = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\Lib"
        purelib = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\Lib\site-packages"
        scripts = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\Scripts"
        stdlib = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\Lib"

Variables:
        BINDIR = "C:\Users\Sascha\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0"
        BINLIBDEST = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\Lib"
        EXE = ".exe"
        EXT_SUFFIX = ".cp310-win_amd64.pyd"
        INCLUDEPY = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\Include"
        LIBDEST = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\Lib"
        SO = ".cp310-win_amd64.pyd"
        TZPATH = ""
        VERSION = "310"
        abiflags = ""
        base = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0"
        exec_prefix = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0"
        installed_base = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0"
        installed_platbase = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0"
        platbase = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0"
        platlibdir = "lib"
        prefix = "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0"
        projectbase = "C:\Users\Sascha\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0"
        py_version = "3.10.6"
        py_version_nodot = "310"
        py_version_nodot_plat = "310"
        py_version_short = "3.10"
        srcdir = "C:\Users\Sascha\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0"
        userbase = "C:\Users\Sascha\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages"
strongly-typed commented 2 years ago

The "simpel_example" from the project already give different results when using msvcrt.dll. So it is not a problem of my DLL.

The code uses pointers and memset

# Add 4 'O' to the string starting from position 6
p = ctypes.cast(ctypes.addressof(mut_str) + 5, ctypes.POINTER(ctypes.c_char))
libc.memset(p, ctypes.c_char(b"O"), 4)

# Print the modified string
libc.puts(mut_str)

The expected result (which is produced by CPython) is XXXXXOOOO. IronPython does not write to the correct memory.

PS C:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\simple_example> $Env:IRONPYTHONPATH="C:\Users\Sascha\Desktop\working_directory\ironpython3\Src\StdLib\Lib"
PS C:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\simple_example> ..\..\ironpython3\bin\Release\net6.0\ipy.exe .\t0.py
Succesfully loaded the system C library from "C:\Windows\system32\msvcrt.dll"
Hello from Python to C
Using the C printf function from Python ...
XXXXX
XXXXX
PS C:\Users\Sascha\Desktop\working_directory\Python-using-C-CPP-libraries-with-ctypes\simple_example> python .\t0.py
Succesfully loaded the system C library from "C:\Windows\system32\msvcrt.dll"
Hello from Python to C
Using the C printf function from Python ...
XXXXX
XXXXXOOOO
strongly-typed commented 2 years ago

The problem also exists on macOS

➜  simple_example git:(master) ~/Dev/ironpython3/bin/Debug/net6.0/ipy t0.py
Succesfully loaded the system C library from "/usr/lib/libc.dylib"
Hello from Python to C
Using the C printf function from Python ...
XXXXX
XXXXX
➜  simple_example git:(master) python3.10 t0.py
Succesfully loaded the system C library from "/usr/lib/libc.dylib"
Hello from Python to C
Using the C printf function from Python ...
XXXXX
XXXXXOOOO
➜  simple_example git:(master) sw_vers
ProductName:    Mac OS X
ProductVersion: 10.15.7
BuildVersion:   19H2026

I can see the following difference between

IronPython

>>> mut_str.__class__
<class 'unknown.c_char_Array_10'>

and CPython

>>> mut_str.__class__
<class 'ctypes.c_char_Array_10'>
strongly-typed commented 2 years ago

At least this code does something expected (on macOS):

import ctypes, ctypes.util
path_libc = ctypes.util.find_library("c")
libc = ctypes.CDLL(path_libc)

x = ctypes.create_string_buffer(10)
z = libc.memset(x, ctypes.c_char(b'X'), 5)

p = ctypes.addressof(x) + 5
# p = ctypes.cast(p, ctypes.POINTER(ctypes.c_char))  # This does break it

libc.memset.argtypes = [ctypes.c_void_p, ctypes.c_char, ctypes.c_long]  # This is necessary
z = libc.memset(p, ctypes.c_char(b'O'), 4)
libc.puts(x)
XXXXXOOOO
10

Also works on Windows:

XXXXXOOOO
0
slozier commented 2 years ago

No idea if it's related, but the simple example issue reminds me of the following (from https://github.com/IronLanguages/ironpython2/issues/653)

import ctypes

class POINT(ctypes.Structure):
    _fields_ = [('x', ctypes.c_long), ('y', ctypes.c_long)]

buf = ctypes.create_string_buffer(ctypes.sizeof(POINT))
ptr = ctypes.cast(buf, ctypes.POINTER(POINT))

ctypes.windll.user32.GetCursorPos(ptr)

print(ptr.contents.x, ptr.contents.y)

If I tweak it to not pass the pointer to the function then it seems to work:

import ctypes

class POINT(ctypes.Structure):
    _fields_ = [('x', ctypes.c_long), ('y', ctypes.c_long)]

buf = ctypes.create_string_buffer(ctypes.sizeof(POINT))
ptr = ctypes.cast(buf, ctypes.POINTER(POINT))

ctypes.windll.user32.GetCursorPos.argtypes = [ctypes.c_void_p]
ctypes.windll.user32.GetCursorPos(ctypes.addressof(buf))

print(ptr.contents.x, ptr.contents.y)

I have a vague recollection that the issue might have been that it was passing the address of the pointer instead of the value of the pointer to the function - but it was a few years ago that I looked at it so I could be wrong. Unfortunately I never came up with a solution.