erickt / rust-zmq

Rust zeromq bindings.
Apache License 2.0
886 stars 189 forks source link

Sockets are Send but not thread-safe #367

Closed jimpo closed 1 year ago

jimpo commented 1 year ago

According to the ZMQ API docs

Individual ØMQ sockets are not thread safe except in the case where full memory barriers are issued when migrating a socket from one thread to another. In practice this means applications can create a socket in one thread with zmq_socket() and then pass it to a newly created thread as part of thread initialization, for example via a structure passed as an argument to pthread_create().

On the page for zmq_socket, the docs say

ØMQ has both thread safe socket type and not thread safe socket types. Applications MUST NOT use a not thread safe socket from multiple threads except after migrating a socket from one thread to another with a "full fence" memory barrier.

In the Rust bindings, however, zmq::Socket implements Send, indicating to the type checker that it is safe to migrate sockets between threads freely. It seems to me that Socket should be !Send. To allow migration between threads, a Socket could be wrapped in some sort of envelope type to be safely sent between threads which does the memory fence before unwrapping back into a Socket.

jimpo commented 1 year ago

Actually, on further investigation, it seems like all the channel constructions that could be used to actually transfer ownership between threads do the memory barrier required by ZMQ.

bkolligs commented 1 year ago

Any additional thoughts on this? I was about to start using a Socket without a mutex but then saw this issue.

bkolligs commented 1 year ago

You second link suggests that there is a thread safe socket type. Does rust-zmq not use those?

jimpo commented 1 year ago

Yeah, my assessment is that it is safe to transfer ownership of Socket through a channel or a std::thread::spawn call because those calls do the appropriate memory fence operations. I am closing this issue.