IronLanguages / ironpython3

Implementation of Python 3.x for .NET Framework that is built on top of the Dynamic Language Runtime.
Apache License 2.0
2.5k stars 287 forks source link

ctypes .from_buffer only works for array.array #1404

Open orent opened 2 years ago

orent commented 2 years ago

The .from_buffer[_copy] methods of ctypes types only accept an array.array argument. Other implementations such as cpython, pypy also accept bytes, bytearray, mmap, memoryview etc.

The struct and re module accept a wider variety of buffer-like objects. Strangely, re accepts mmap objects, but stuct does not.

orent commented 2 years ago

Correction: from_buffer_copy() does accept bytes, but not bytearray, memoryview, etc.

BCSharp commented 2 years ago

Thanks for the submission. This seems to be related to #21, which is only partially finished. What is still needed is a review of all modules and converting their API to use IBufferProtocol as needed. struct and ctypes are the two prominent ones. Especially getting ctypes up to speed will be exciting because it is a prerequisite for supporting numpy.

slozier commented 2 years ago

from_buffer is probably blocked by https://github.com/IronLanguages/ironpython3/issues/1297. from_buffer_copy should be simple to fix with IBufferProtocol (thanks to @BCSharp).

BCSharp commented 2 years ago

Yes, from_buffer_copy will be simple. I can pick it up since the int/long unification stuff is drying up. For the from_buffer, indeed #1297 needs to be done first.

slozier commented 2 years ago

@BCSharp I guess I should have waited a bit! https://github.com/IronLanguages/ironpython3/pull/1405 could use a review.

BCSharp commented 2 years ago

No problem! I wouldn't be able to get at it till tonight at best. Will review your PR instead, hopefully this time pushing all the right buttons... 😄

orent commented 2 years ago

Thanks for the fast response. Two more minor inconveniences that caused my code not to work out-of-the-box on IronPython are:

  1. struct.unpack_from not accepting an mmap object
  2. ctypes.Structure._pack_ attribute accepts only int, not bool

Should I open issues for these?

what is the easiest way to get a build of something newer than 3.4alpha?

slozier commented 2 years ago

Should I open issues for these?

I think struct.unpack_from will take a bit of effort to implement properly and might be worth its own issue. I can take a look at ctypes.Structure._pack_ later today, I think it should be easy.

what is the easiest way to get a build of something newer than 3.4alpha?

You should be able to get the latest build from the main page. If you search for "builds" there are badges for the Azure and Github CI. They should both publish build artifacts that you can download.

BCSharp commented 2 years ago

After giving it some thought I am now becoming more convinced that #1297 is not the way to go here. In the coming days I will do some experiments for an alternative solution (like pinning supported by IPythonBuffer). If it goes somewhere I will submit a draft PR to elicit comments.

orent commented 2 years ago

It's confusing, because all the newly built packages are still named "3.4alpha", just like the April 2021 release.

On Mon, 18 Apr 2022 at 16:09, slozier @.***> wrote:

Should I open issues for these?

I think struct.unpack_from will take a bit of effort to implement properly and might be worth its own issue. I can take a look at ctypes.Structure.pack later today, I think it should be easy.

what is the easiest way to get a build of something newer than 3.4alpha?

You should be able to get the latest build from the main page. If you search for "builds" there are badges for the Azure and Github CI. They should both publish build artifacts that you can download.

— Reply to this email directly, view it on GitHub https://github.com/IronLanguages/ironpython3/issues/1404#issuecomment-1101395628, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF2ID3NNWVWCUCEBGHDZKTVFVNJJANCNFSM5TUADXRA . You are receiving this because you authored the thread.Message ID: @.***>