Closed Avasam closed 2 years ago
The Python wrappers for IBuffer
and IMemoryBuffer
implement the CPython buffer protocol, so they can usually be passed directly to other Python APIs that expect a bytes-like object.
If I try to use directly with numpy, I get the following error:
bitmap_buffer = software_bitmap.lock_buffer(BitmapBufferAccessMode.READ_WRITE)
np.frombuffer(bitmap_buffer, dtype=np.int8)
a bytes-like object is required, not '_winsdk_Windows_Graphics_Imaging.BitmapBuffer'
It looks like in the case of IMemoryBuffer
you need to call create_reference()
and then pass the return value from to numpy.
I was so close, thanks for steering me in the right direction!
software_bitmap = async_operation.get_results()
reference = software_bitmap.lock_buffer(BitmapBufferAccessMode.READ_WRITE).create_reference()
image = np.frombuffer(cast(bytes, reference), dtype=np.uint8)
Is it not possible without the cast? I guess that just make the linter happy?
software_bitmap = async_operation.get_results()
buffer = software_bitmap.lock_buffer(BitmapBufferAccessMode.READ_WRITE)
image = np.frombuffer(buffer.create_reference(), dtype=np.int8)
The cast is just for the linter when type checking, from https://github.com/numpy/numpy/blob/main/numpy/__init__.pyi#L1433:
# There is currently no exhaustive way to type the buffer protocol,
# as it is implemented exclusivelly in the C API (python/typing#593)
_SupportsBuffer = Union[
bytes,
bytearray,
memoryview,
_array.array[Any],
mmap.mmap,
NDArray[Any],
generic,
]
(I've also updated my previous message with dtype=np.uint8
)
It is fully working, I've been able to render the image in my application.
I can get from a Direct3D11CaptureFrame to a SoftwareBitmap like so:
According to the official Microsoft doc, I could call LockBuffer, then CreateReference and finally GetBuffer to obtain a bytarray (https://docs.microsoft.com/en-us/windows/uwp/audio-video-camera/imaging#create-or-edit-a-softwarebitmap-programmatically) But when I do
reference = software_bitmap.lock_buffer(BitmapBufferAccessMode.READ_WRITE).create_reference()
,reference
has no methodget_buffer
. (maybe it's missing a binding? Also not sure that byte-array is even meaningful here)The doc does have a section about integration with OpenCV: https://docs.microsoft.com/en-us/windows/uwp/audio-video-camera/process-software-bitmaps-with-opencv#3-implement-the-opencvhelper-class I believe opencv2-python's equivalent of
mat
used to becv2.CreateMat
, which I believe is now simply using a numpy array. But I'm once again stuck trying to obtain thepPixelData
, which is obtained fromIMemoryBufferByteAccess::GetBuffer method