search

justinstenning / SharedMemory

C# shared memory classes for sharing data between processes (Array, Buffer and Circular Buffer)
https://www.nuget.org/packages/SharedMemory
Other
566 stars 118 forks source link

Multiple Readers and Writers and buffer name discovery #7

Open MikeFair opened 8 years ago

MikeFair commented 8 years ago

I am researching integrating this library as the IPC mechanism for the NetMQ project (DotNet ZeroMQ project - also here on gh) and am looking for how to manage multiple readers and writers for that scenario.

What would be ideal is having each file participant (when there are multiple readers) be able to track which nodes they've read from the buffer independently (which I think means something like they each have their own separate read/write pointers looking at the same shared buffer data); copy the data as few times as is possible/necessary (again, using separate node lists, but referencing the same shared memory data); bi-directional communication (one memory file per "writer"?); and some way to easily find out if/when a participant has written data to the channel.

Right behind that would be discovering when a new Participant has connected to the channel. Is there some kind of eventing that could be used for this? I'd thought about using a MessageOnly Window Class and sending the Node Address Pointer (which exists in the shared memory region) as the message payload. That way all the readers would get a separate message to add/append the node address to the end of their queue. Then it only requires a scheme to know when the node could be deleted/reclaimed while dealing with aborted/blocked processes that will never process the message and/or read the node block...

I'm still working through the details, but ideas are welcome.

Thanks

justinstenning commented 8 years ago

Not really directly related to the "multi-channel" idea you have here, but if there are going to be changes in this area it might be worth also considering the following - or at least keep in the back of mind. The same idea of using multiple read pointers was put forward too: Allow all readers to read all the nodes: see issue #8

justinstenning commented 8 years ago

Thinking on this further, you could indeed probably use the read/write pointers as the channels, if so I do think it worth supporting the two approaches, e.g. channel and also a single producer + broadcast to multiple readers.