Provide memory efficient access to image data

[matham]

Issue I got over email:

I have a specific question regarding the Image.get_image_data function:

It appears that the Image.get_image_data function creates a copy of the underlying image buffer, which costs our tracking system a considerable amount of computing performance. Do you think it would be possible to get direct access to the underlying buffer pointer without the need for copying? Perhaps some form of memory view? Alternatively, it would work if the function could accept a pointer to an empty NumPy array to fill.

[matham]

My reply:

What you describe is certainly possible, I didn't implement it because in my usage I hadn't yet had a need to optimize these lower level stuff.

There are two approaches, as you suggested. One, is to return the original pointer as a memoryview. I hadn't done this initially, because then you have to worry about the user still accessing the pointer through the memory view after the image and its data is released, potentially causing a seg fault in python. I have done something like this here: https://github.com/matham/ffpyplayer/blob/master/ffpyplayer/pic.pyx#L803.

The other approach is for the user to provide a memory view into which we copy the data into, such as a numpy array or bytesarray. It wasn't clear to me however, how much this saves because it'd depend on their workflow.

I would have liked to do some performance testing. Because I suspect that just creating the Image object is also what costs a lot of time. Did you check performance that get_image_data specifically is where the performance issues happen and not e.g. when getting the image from the camera?

[matham]

I did some profiling and indeed it does seem to be an issue:

from timeit import timeit
from rotpy.image import Image

empty_img = Image.create_empty()
full_image = Image.create_image(640, 480, 0, 0, 'Mono16')

setup = '''
from rotpy.image import Image
from __main__ import empty_img, full_image
'''
stmt_empty = '''
image = Image.create_empty()
'''
stmt_full = '''
image = Image.create_image(640, 480, 0, 0, 'Mono16')
'''
stmt_get_data = '''
data = full_image.get_image_data()
'''

print(f'Empty image: {timeit(stmt=stmt_empty, setup=setup, number=1000):f} sec / 1000')
print(f'Full image: {timeit(stmt=stmt_full, setup=setup, number=1000):f} sec / 1000')
print(f'Get data: {timeit(stmt=stmt_get_data, setup=setup, number=1000):f} sec / 1000')

prints:

Empty image: 0.001595 sec / 1000
Full image: 0.018645 sec / 1000
Get data: 0.501147 sec / 1000