nanomsg / nnpy

cffi-based Python bindings for nanomsg
MIT License
117 stars 39 forks source link

Allow passing memoryview/buffers to send #14

Closed memeplex closed 8 years ago

memeplex commented 8 years ago

As nanomsg doesn't provide multipart messages it's convenient to create a BytesIO and dump a sequence of pickles into it. That way the "parts" of the message can be recovered at the other side by means of an equivalent sequence of pickle loads.

But the current implementation of Socket.send in nnpy requires the memoryview associated with the BytesIO to be transformed to bytes first:

TypeError: initializer for ctype 'char[19]' must be a bytes or list or tuple, not memoryview

This is not efficient, as it implies an extra copy of the message. But it seems the cffi guys implemented a conversion from buffer to char array:

https://bitbucket.org/cffi/cffi/issues/65/support-memoryview-to-char-conversion https://bitbucket.org/cffi/cffi/issues/47/creating-a-cdata-from-a-buffer

I think it's important to support this in nnpy, as nanomsg is intended to be fast and require as little copy as possible.

memeplex commented 8 years ago

I don't have the time to create a PR just right now but this seems to work fine in python 3:

def send(self, data, flags=0):
        l = len(data)
        if isinstance(data, memoryview):
            buf = ffi.from_buffer(data)
        else:
            data = data.encode() if isinstance(data, str) else data
            buf = ffi.new('char[%i]' % l, data)
        rc = nanomsg.nn_send(self.sock, buf, l, flags)
        assert rc > 0, rc
djc commented 8 years ago

Thanks for reporting and the fix!