search

asajeffrey / shared-data

A Rust library for data structures living in shared memory.
Mozilla Public License 2.0
12 stars 2 forks source link

Added count method to SharedRc #9

Closed jean-airoldie closed 5 years ago

jean-airoldie commented 5 years ago

This allows the user to retrieve the current reference count.

asajeffrey commented 5 years ago

Thanks! BTW, are you using shared-data for something, or just being a good citizen?

jean-airoldie commented 5 years ago

I'm thinking of using it to do IPC via some a async channels based on tokio::UnixDatagram.

asajeffrey commented 5 years ago

Oooh, let me know how you get on! shared_channel has as far as I got with channels, but I've not looked into async. I suspect it'd need a custom executor, as the OS condition variables aren't async-aware.

jean-airoldie commented 5 years ago

Yeah I looked into a shared async channel and I found that its not really realistic atm. I'm using unix domain sockets instead. Also I don't think I can use your crate because I would need deallocation. I'm thinking I might have to implement a mmap allocator so that I can use these primitives.

asajeffrey commented 5 years ago

The problem we've been having in servo with domain sockets is running out of fds, since ipc-channel creates an fd every time a channel is sent on a channel (there's a use of dup() under the hood). One of the goals of this crate is to put a fixed (and small!) bound on the number of fds used.

Deallocation is a big TODO item, the crate's not really usable until that's in place. I did look at the alloc traits, IIRC the problem I hit was that Box<T, Shmem> will be a pointer to shared memory, in the current process's address space, so it can't be sent using shared memory.

jean-airoldie commented 5 years ago

since ipc-channel creates an fd every time a channel is sent on a channel (there's a use of dup() under the hood).

My approach was essentially to make sure that every socket drain its buffer when it is closed. This way I can simply send a RawFd without the fear of leaking it. Specifically I convert the channel into a owned intermediary representation that contains the RawFd then I proceed to send it.

IIRC the problem I hit was that Box<T, Shmem> will be a pointer to shared memory

That wouldn't be a problem for a shitty mmap allocator that directly allocates from the OS instead of using a memory pool like you did, which I might end up using temporarily if the performance is not too bad.

Another possible less shitty approach would be to allocate large blocks ahead of time, put them in a pool, but don't reuse memory (i.e. when memory is deallocated, it is not returned to the pool).