imglib / imglyb

Connecting Java/ImgLib2 + Python/NumPy
https://pypi.org/project/imglyb/
BSD 2-Clause "Simplified" License
31 stars 5 forks source link

Use scyjava converters for zero-copy wrapping of array-backed imgs to python #20

Closed hinerm closed 1 year ago

hinerm commented 2 years ago

The built-in scyjava array converters will eventually be written to use python memoryview, allowing for zero-copy wrapping of primitive Java arrays. It seems sensible that we would use this logic in imglyb to convert compatible Imgs in the Java -> python direction.

hinerm commented 2 years ago

tagging @elevans @ctrueden

ctrueden commented 2 years ago

For now, I did this work (for raw primitive arrays only, not for ImgLib2 images yet) as a PR in pyimagej: imagej/pyimagej#227. But migrating the code into imglyb would be great.

ctrueden commented 1 year ago

As discussed in scijava/scyjava#53, memoryview does not work to access Java primitive arrays from Python without copying. It supposedly might not copy depending on the JVM implementation and memory layout for that particular array, but in our tests: A) it does make a copy; and B) multiple memoryviews on the same array use the same copy even if the Java array changes in between. So using memoryview did not end up being viable in scyjava. We are now using bytearray instead to just explicitly copy, once, and retain read-write capabilities on the Python copy of the array. For non-huge arrays at least, this is fine, and we do not have a strong use case for huge primitive arrays versus other solutions (e.g. Python-side or Java off-heap/direct memory allocation).

So this issue here can be closed, since scyjava does now support conversion of Java primitive arrays to Python in an efficient way. It does require numpy as a dependency, but optionally—if numpy is not installed then a simple Python list is created, translating the elements one by one (which is slow but functional).