Zero-copy support

[siriux]

Is it possible to do zero-copy with ipc-channel today? I've seen OsIpcSharedMemory, but I'm not sure it can be used directly for this purpose.

If it's not possible right now, do you plan to support it? I'm testing some ideas for interprocess video transfer, and zero-copy would be really helpful.

Thanks

[emilio]

I think this was discussed, and the conclusion was something like: Even though it's hard to expose a safe API on top of shmem to avoid races, letting the user write to a shared memory buffer with an unsafe API could be desirable, but it's just not done.

It should be a straight-forward addition I think, a patch would probably be welcomed :)

cc @antrik, did I get that right?

[siriux]

I'm quite new to Rust, but I might try to write a patch when I get more confidence. Thanks

[antrik]

We need to clarify the use case here.

What IpcSharedMemory currently allows, is creating a shared memory object initialised with some pre-existing data, and sending that to other processes, where it is mapped, so the data can be read there. This is not zero-copy, since creation of the object allocates new memory that is initialised by copying some existing data.

This could be fixed by extending the API to allow decoupling the allocation of the buffer from initialisation. This way we could create a new buffer for each frame, then fill it with data, and finally send it to the consumer process, where it is read and finally freed.

One possible approach for that would be introducing some kind of proto-SHM-object, which is allocated but not initialised, and can be borrowed mutably to fill it with data, before it's converted into a regular immutable SHM object for sending. Another approach would be allowing some kind of callback for supplying the initialisation data -- this would be less flexible in how the data can be supplied (probably precluding DMA?), but keep the API entirely safe, by not exposing uninitialised memory to users.

An entirely different case would arise if we want permanent shared memory, that can be overwritten by the producer process with new data repeatedly even after sending it off to a consumer process. This would be much more involved, since it requires some kind of inter-process locking to make it safe. Wrapping this in some kind of safe abstraction might be somewhat tricky I suspect, though I haven't spent too much thought on it. (And I'm not really familiar with inter-process locking mechanisms.)

[siriux]

Thanks Olaf.

What I had in mind was more or less what you described, borrowing mutably to initialize directly on the shared buffer, as it's more flexible. But if you want to be fast, I think some kind of reusing is needed too.

In my opinion, the easiest way would be to keep the same simple interface (+ mutable borrow), where every time you borrow a new mutable buffer it can contain any data (or is zeroed), but you try to be smart inside and reuse the shared buffers as much as possible.

One "simple" way of doing this is to reuse a buffer that was sent to you, and was already used on your side (dropped?), to send it back with new data. Of course, you have to take care of many details: you need to keep a pool of used buffers, if you don't have a buffer available (or if it's not big enough) you need to create a new one, if you have too many buffers in the pool you need to send them back empty to fill the other side pool (uneven traffic), you can keep track of the largest used size over time and discard large buffers when needed, ...

This approach wouldn't require explicit locking, it's implicitly handled passing buffer ownership in the side channel.

What do you think?

On Tue, Nov 29, 2016 at 3:23 AM, Olaf Buddenhagen notifications@github.com wrote:

We need to clarify the use case here.

What IpcSharedMemory currently allows, is creating a shared memory object initialised with some pre-existing data, and sending that to other processes, where it is mapped, so the data can be read there. This is not zero-copy, since creation of the object allocates new memory that is initialised by copying some existing data.

This could be fixed by extending the API to allow decoupling the allocation of the buffer from initialisation. This way we could create a new buffer for each frame, then fill it with data, and finally send it to the consumer process, where it is read and finally freed.

One possible approach for that would be introducing some kind of proto-SHM-object, which is allocated but not initialised, and can be borrowed mutably to fill it with data, before it's converted into a regular immutable SHM object for sending. Another approach would be allowing some kind of callback for supplying the initialisation data -- this would be less flexible in how the data can be supplied (probably precluding DMA?), but keep the API entirely safe, by not exposing uninitialised memory to users.

An entirely different case would arise if we want permanent shared memory, that can be overwritten by the producer process with new data repeatedly even after sending it off to a consumer process. This would be much more involved, since it requires some kind of inter-process locking to make it safe. Wrapping this in some kind of safe abstraction might be somewhat tricky I suspect, though I haven't spent too much thought on it. (And I'm not really familiar with inter-process locking mechanisms.)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/servo/ipc-channel/issues/126#issuecomment-263457765, or mute the thread https://github.com/notifications/unsubscribe-auth/AArqenAgWEqrxKIKrKkcCU_d5pb8Jw3Fks5rC4yEgaJpZM4K9qr9 .

[antrik]

Let's not over-engineer this. The more flexible variant is not necessarily better, if it requires an unsafe interface. Do you actually need this flexibility for your use case? Please be more specific about what you are trying to do.

Same regarding buffer reuse. Of course we could synchronise the use of mutable buffers using only IPC channel messages, rather than adding some other cross-process synchronisation mechanism -- but that's likely less efficient. At any rate, if we try to reuse buffers, we definitely need some kind of cross-process synchronisation, which is always fairly expensive -- so it has to be very carefully considered, to make sure it's actually an optimisation... Again, we need specifics.