Tracking issue for channel selection

[alexcrichton]

This is a tracking issue for the unstable mpsc_select feature in the standard library. Known points:

• The API is unsafe, and should probably never be stabilized.
• Servo apparently makes very heavy use of this API, so it can't be removed without replacement currently.
• The correctness of the implementation is somewhat dubious, it's incredibly complicated.
• The macro-based approach for selection is dubious.
• This is not a scalable implementation of selection (e.g. "poll everything each time wait() is called")
• This drags down some other channel-related code with a few mutexes in otherwise lock-free code.

[jdm]

cc me

[steveklabnik]

Is it worth investigating @BurntSushi 's https://github.com/BurntSushi/chan ?

[steveklabnik]

I also think @carllerche was supporting the idea of removing channels from the stdlib

[alexcrichton]

Note that I'm not advocating removing channels from the standard library (plus they're stable). Removing selection is also probably not an option as it's so closely tied to the implementation details of channels.

One of the major points of complexity of the current implementation is that it's all basically 99% lock free (giving us a nice performance boost in theory). I believe @BurntSushi's implementation (please correct me if I'm wrong!) is largely based on mutexes and condition variables.

[BurntSushi]

@alexcrichton Correct. I doubt very much that chan will ever be lock free because I'm not convinced the semantics I want can be done with them. It's also much easier to implement. :-)

[nagisa]

I personally prefer the epoll-like design as implemented in comm.

[carllerche]

Streams in Eventual are lock free :) There is also stream selection, which is lock free (and definitely is a pain to implement).

Re: removing channels from stdlib, as @alexcrichton points out, they are stable. I would not worry about adding select to them. They are good as a starting point. If somebody requires more complex concurrency semantics, they can use a lib. I'm not sure why @alexcrichton thinks that removing select from std is hard though...

[tripped]

Just hopping on to mention #12902 as one of the current weaknesses of the macro approach to select!.

[softprops]

Fwiw, channel selection with golang channels is part of what makes golang concurrency story so alluring to newcomers to go. For a newcomer to a safe concurrent language like rust, needing to mull over which 3rd party concurrency crate is popular this week to pick for actually writing concurrent code feels onerous. Having primatives like channels and channel selection in the std lib make rust more attractive for concurrent applications than alternatives like go and make crates a little more interoperable than resolving conflicting implementation issues betwwen 3rd party options.

[BurntSushi]

Fwiw, channel selection with golang channels is part of what makes golang concurrency story so alluring to newcomers to go.

@softprops This doesn't address your primary concerns, but FYI, chan_select! in the chan crate was built to specifically match the semantics of Go's select.

[softprops]

@burntsushi nice!

[jonhoo]

Correct me if I'm wrong, but doesn't the macro approach also have the drawback that selecting over a dynamic set of channels isn't easily expressible?

[szagi3891]

Best to get rid of the macro select. I made a working prototype that shows that it is possible: https://github.com/szagi3891/channels_async-rs/blob/master/examples/select.rs#L76 Without any section unsafe.

[canndrew]

Here's a wacky idea: add support for generically sized tuples and anonymous enums. Then it will be possible to select across an arbitrary number of channels of different types without macros:

let ch0: Receiver<i32> = ...;
let ch1: Receiver<String> = ...;
match select((ch0, ch1)) {
    (x|!) => println!("Got an i32: {}", x),
    (!|y) => println!("Got a String: {}", y),
}

[BurntSushi]

Is there anyone still using this API? I don't see any obvious path to stabilization. It would be nice to just remove it.

[alexcrichton]

@jdm is this still being used in Servo?

[SimonSapin]

grep finds 4 instances of select! in servo master.

[mattgreen]

I'd like to remove this from watchexec, but I'm unsure what to move to. One of my crates already streams data to me via a channel, so I hopped onto that bandwagon.

I'm usually watching for either control-c or file system inputs at all times, since I have cleanup to do when the user hits control-c. select! might be unstable but I can at least accomplish this.

[softprops]

I've had only good experiences with https://github.com/BurntSushi/chan

[szagi3891]

@mattgreen : I think you should be able to replace this selecta! macoro of one channel.

This channel type will be :

enum Message {
    CtrlC,
    NormalMessage(....),
}

The main thread would send in theright time message Message::CtrlC. The remainder of the program would send messages Message::NormalMessage

It makes sense ?

[antrik]

Servo's ipc-channel library ( https://github.com/servo/ipc-channel ) also uses mpsc_select (for its inprocess back-end) -- which is a bit of a pain, since platforms relying on this back-end can't use ipc-channel with stable Rust. (See https://github.com/servo/ipc-channel/issues/118 )

It should be noted though that this back-end is not a real implementation, but rather just a partial workaround for platforms that don't have a "real" inter-process implementation yet. While it was originally created for Windows, most likely Windows will be getting a proper implementation soonish. ( https://github.com/servo/ipc-channel/pull/108 )

Right now the inprocess back-end is also being used for Android -- though I'm not sure why really...

So while this back-end might continue to be useful during bring-up of new platforms (depending on whether Servo keeps the option of running in a single process), it's not exactly a first-class citizen I'd say.

[antrik]

I'd like to point out that https://github.com/BurntSushi/chan -- which is being touted as a replacement for mpsc here -- doesn't really seem like a valid alternative to me: unlike mpsc, it has clonable and sharable receivers.

This might seem convenient for some use cases; but I'm not convinced it's a real win over just wrapping the receiver in a Mutex<> or Arc<Mutex<>> yourself when you know it's save and necessary for your application. (As mutexes are safe and easy to use in Rust -- unlike in Go -- I don't see any motivation for channels to double as a locking substitute...)

More importantly though, for the vast majority of use cases, sharing the receiver is actually not desirable: rather, it often indicates bugs -- and thus not rejecting this by default is indeed more error prone!

(Not even considering implementation overhead for something that's not needed at all in most cases...)

[BurntSushi]

@antrik Please see the README for chan. Notably:

The purpose of this crate is to provide a safe concurrent abstraction for communicating sequential processes. Performance takes a second seat to semantics and ergonomics. If you're looking for high performance synchronization, chan is probably not the right fit.

There are a number of reasons why it doesn't necessarily replace std::sync::mpsc. Nevertheless, if you want to write in a CSP style in Rust, then I do think it's your only option on stable Rust because it gives you chan_select!. I suspect that's why people are bringing it up (which seems reasonable to me).

[antrik]

@BurntSushi as long as there is a general understanding that chan is a special-purpose library rather than a general replacement, that's perfectly fine -- I'm just concerned that there seems to be an air here of "we don't need to fix mpsc since there is chan", which I find rather disconcerting...

[antrik]

To be perfectly clear, my point is that I believe there is a genuine need for mpsc to have a stable select; or failing that, at the very least for some other library that can indeed serve as a real drop-in replacement. (And is prominently featured as such -- I for one still think that the prevalent "go look around for some random crate that seems like it might fit this very basic need" mindset is seriously hurting the Rust ecosystem...)

[BurntSushi]

I don't think anyone's really contesting that though? I think these are the facts on the ground as I know them:

• mpsc_select hasn't seen any attention towards stabilizing it. I don't think there's even a viable proposal in existence at all.
• It seems the only thing keeping mpsc_select around is Servo.
• Some have found success by switching to chan. Heck, the fact that mpsc_select was unstable was one of the reasons why I wrote chan in the first place!

[p-kraszewski]

As a Rust newcomer from Erlang/Elixir world - I can live with MPSC.

My question is - do you have any comments on @szagi3891 approach for a static list of channels? Instead of selecting on let's say 2 channels, just .receive on one, carrying enum that is matched and dispatched later? Any do-s, don't-s, don't care-s?

[barr1969]

Select on Dynamic Lists of Channels

As someone whose been using CSP for decades (though I've not yet used Rust), I feel compelled to submit a plea for the ability to select across dynamic lists of channels. This is such a useful thing to be able to do. It's particularly useful in large systems that have to deal with failure. That sounds like I'm effectively supporting @canndrew's suggestion. Lists that are fixed at compile time diminsh what can be achieved.

Locks, Why They're Not a Big Problem, and Program Architecture

On the topic of striving for a lock free implementation, I'm not convinced there's much merit in that if it results in too much ugliness in the implementation, pain, grief, anguish, etc.

If a receiver is selecting across one or more channels then it is implicitly content to wait for messages to arrive. So there's no real penality there from the receiver's point of view.

However, if the receiver never ends up waiting because the flow of messages exceeds its ability to process them, and this is seen as a performance problem (where a lock-free implementation might be seen as a way of improving performance), then my argument is that one's program's architecture is inadequate. The implication is that the receiver is a performance bottle neck.

The real solution is that there needs to be additional receivers, a separate channel to each one, and a means (e.g. either round robin, push pull, etc - the parallels with some of ZeroMQ's patterns and load balancing are not coincidental) for the sender to know to which one the next message should be sent.

The ideal architecture is one where there's enough parallel receivers to juuuuust ensure that the sender never has to block. The sender should always be finding that a receiver is available to receive a message.

The Actor model is often favoured because of its simplicity, but the runtime problems that might occur (deadlock, livelock, etc), are unpleasant. By using a strict CSP model, and getting the architecture just right, nothing ever blocks on send (just like Actor) but still achieves rendevous but with the added advantage of fixed latencies in data flow through your program. That's generally nice for real time things like streaming video, meeting performance targets, etc.

Now, if locks are used as part of the channel implementation, then the sender's end of a channel should then always be finding that the locks are available (because there's enough receivers to keep up with the required data flow rate). And if the locks are always available, they're probably quick to acquire.

So I'm basically suggesting that we probably shouldn't worry about an implementation using locks; if the locks are thought to be a problem, one's application architecture is likely to be more of an issue than the overhead of the locks.

[jonhoo]

As a separate suggestion, the eventual select interface should also have a try_recv() equivalent, as well as a recv_timeout(). I currently call try_recv() manually on each channel a couple of times before calling wait() to avoid incurring a thread sleep/wakup cycle, and being able to instead call try_recv directly on the Select would be much better.

[jonhoo]

Another use-case that's been bothering me lately is if I have multiple channels, some of which are likely to always be non-empty. The current Select implementation does not give any kind of guarantee of fairness, and if the first channel mentioned in the Select always has an element available, the other channels will never be read from.

EDIT: For reference, Go chooses the next channel to try in a select pseudo-randomly.

[adrianbrink]

Is this an unstable feature that I can already use on nightly?

[steveklabnik]

yes https://doc.rust-lang.org/stable/std/macro.select.html

[bstrie]

Those following this thread will probably be interested in https://github.com/crossbeam-rs/rfcs/pull/22 .

[ghost]

crossbeam-channel has been just released: https://docs.rs/crossbeam-channel

This crate aims to be a faster and more featureful alternative to std::sync::mpsc. I believe it should basically tick the boxes of all requests that have been brought up in this thread. I encourage everyone to give it a shot and let me know what they think about it. :)

---

Regarding std::sync::mpsc, I must agree with @carllerche here:

Re: removing channels from stdlib, as @alexcrichton points out, they are stable. I would not worry about adding select to them. They are good as a starting point. If somebody requires more complex concurrency semantics, they can use a lib.

Implementing fast channels is difficult enough, and adding selection on top of them makes implementation crazy complicated. Perhaps the standard library should offer very simple channels (without selection) that work as a starting point, deferring advanced use to external crates?

Honestly, I believe a better fit for the standard library would be a new simple MPMC channel without selection that internally uses locks and conditional variables. It'd support blocking and non-blocking send/recv operations, that's all.

Alternatively, the async variant could use what we have in shared.rs (a simple and pretty fast concurrent queue), and the sync variant could use crossbeam-channel's array-based queue (which even the standard library used to have long time ago). This solution could be performant and not terribly complicated.

My impression is that the present design of std::sync::mpsc can nowadays be seen as a historical artifact that reflects the needs of Servo.

For example, asynchronous channels are highly optimized and have complex implementation, yet the synchronous variant got little attention - it is comparatively simple and much slower. This is probably because Servo needs fast async channels but doesn't care about sync channels at all.

Furthermore, selection is imperfect in a number of ways. There doesn't seem to be a hopeful path towards stabilization of selection, but it has been kept alive (in a permanent unstable state) for years because Servo still depends on it.

[SimonSapin]

It seems the only thing keeping mpsc_select around is Servo.

I’m hoping to move Servo to using crossbeam-channel. If all goes well and that lands, I’ll propose deprecating mpsc selection in std in order to find out if this quote above is accurate.

[droundy]

I'd like to add that I'd like to eventually see a nice channel implementation in the standard library. They can be a part of APIs and once a good select is enabled it really helps for everyone to use the same kind of channels.

[ssokolow]

@droundy The interoperability convention that seems to be developing in the Rust ecosystem is to standardize a set of types and traits, publish them to crates.io alone, and then have each implementation depend on them.

See, for example, the http crate, which provides shared definitions for types like Request and Response.

[WiSaGaN]

@ssokolow I agree with this, and I think this is compatible to have default implementation in standard library. Although comparing to http, I feel channel interface is more fundamental to programming in general (golang has it baked in the language), thus should be in standard library at least eventually.

[jonhoo]

I agree that the channel trait probably should, if we decide to go down this route. In a sense it's almost like Read and Write. The place where it gets tricky is if you want to support something like select across a common API. That I think will be really difficult, if not impossible. But basic channel behaviour I totally agree fits into that paradigm nicely.

[stacktracer]

As a random lurker, I hope I'm not speaking out of turn ... but to me select is the magic sauce that makes channels compelling -- the one thing about channels you can't easily get in other paradigms.

Channels without select ... meh.

[ExpHP]

Just noting here that I didn't bother with select! in #48056 because it seemed to me like it had ergonomical issues that extend well above and beyond its lack of trailing comma support, and that this is an inappropriate problem to be solving with a public macro_rules! macro in libstd. I'm glad to see that stabilization is already considered more or less off the table.

[BatmanAoD]

@alexcrichton Why does the top-level post say that the API "should probably never be stabilized"? Is there some theoretical reason why select can't be done in an acceptable way?

[SimonSapin]

@BatmanAoD I think what was meant is that the API should not be stabilized in its current form.

Designing a better API is possible, arguably https://docs.rs/crossbeam-channel/0.2.3/crossbeam_channel/macro.select.html has done so. Now a logical next question would be: should this or some other channel selection API be added to the standard library? I’d say that it should not, and it’s better to suggest users try crossbeam-channel (or some other library) instead, since libraries on crates.io that can evolve independently from the language.

std::sync::mpsc exists today mostly because it was inherited from once upon a time before Rust 1.0, when Cargo or the Send trait did not exist yet. When green threads and channels were considered a core part of Rust’s concurrency story and were both built into the language. If it were proposed for addition today it would probably not be accepted.

[gterzian]

As someone who is relatively new to Rust, concurrency, or channels, I would like to share my own experience using Select and the broader std::sync::mpsc API.

All I know about channels is what I have learned contributing to Servo for almost two years now. I can say the following with certainty: If it wasn't for std::sync::mpsc, and Select, after two years I would still be scratching my head about how the various components in Servo communicate with each other.

With regards to the internal implementations of channels, my knowledge is limited, let's just say that I was recently surprised to read that there was a lock inside of them...

However, I can tell you that for a newcomer, having a concept like std::sync::mpsc at your disposal is great.

Here are the reasons:

1. It has "fast, async" channels, which are simply much easier to get started with than any blocking flavor.
2. It's in the standard library, so you feel confident to use them and you don't need to shop around.
3. It isn't a mpmc or some other more complicated flavor of channel. To be convinced, just google "Go concurrency patterns".

And as you learn more about how to use them, you come to realize that there is a lot you can do with an mpsc channel, including stuff that you'd thought would require a dedicated flavor.

For example spmc, who says it has to be a single channel? What about having a channel for each of your consumer, who will simply share the Sender half to the producer? Using a set of spsc channels, you can build your own spmc one(Incidentally, it seems that such spsc use of std::sync::mpsc is also the most efficient way to send messages these days).

Regarding mpmc, I'm assuming you'd want to have some business logic around such communication, so why not have a custom 'broker' component and implement the communication with a bunch of std::sync::mpsc channels? In some ways, Servo's constellation could be seen as a giant mpmc channel.

---

Is Select currently really that bad, or missing features? To me, the Go 'more complicated' variant is basically just another way to deal with blocking channels. Who needs to select over a send, when it can never block? On the other hand, Select is certainly useful when you need to receive a message from different components, each represented by a Receiver. It is playing a big part in making logic like this easier to understand.

I would actually go as far as saying that the limited set of features of the current Select is a strength, and that it fits perfectly with the "fast, async" channel API.

And regarding Sender or even Receiver needing to be Sync, it appears to me that in fact almost any communication pattern with a std::sync::mpsc can be implemented without the need for those two to be Sync. I wrote a blog post about it last week.

Also, you need to appreciate the, perhaps incidental, beauty of the design of something that "if it were proposed for addition today it would probably not be accepted". For example using a Receiver as an iterator, allowing a component to 'quit' simply when all it's senders have been dropped elsewhere. To see how hard that is to do with a mpmc channel, just read this https://blog.golang.org/pipelines (here is my version ).

---

std::sync::mpsc is an example of a simple approach to channels that is easy to work with, provides the building blocks for more complicated stuff, and that has been battle-tested in a large and complex project like Servo.

Couldn't we move toward a stable Select, as opposed to the direction of deprecation? I guess there are perhaps some problems with the implementation, and I'm basically arguing in long form that the API is just fine.

I'd do the work myself if I could actually wrap my head around all that manual bookkeeping(that I am so glad my own code using channels doesn't have to deal with). Let me know if I can help in some other way...

[BurntSushi]

@gterzian I found it hard to follow your comment, but have you seen crossbeam_channel? A lot of thought went into that API. It would be interesting to see whether Servo could migrate to it. If not, why not?

[SimonSapin]

Servo has an open PR to move to crossbeam-channel: https://github.com/servo/servo/pull/19515. It is currently blocked on a bug that needs tracking down (and causes tests to fail).

[gterzian]

@BurntSushi What was hard to follow about my comment? Or was it just too long? I have to admit it is blog post length. :)

The TLDR of my comment is this: std::sync::mpsc is absolutely awesome, and Select plays a part in that. I am happy with "fast, async" channels and only selecting over receivers. I also think one can build flavors other than mpsc using mpsc as a building block, so I don't need other flavor of channels or a Select implementation that would support more complicated operations.

I think crossbeam looks interesting, and it's selling points are feature I don't feel are missing from std::sync::mpsc. I would be saddened if the standard library were to lose a full-featured channel implementation.

[gterzian]

@SimonSapin What was the initial impetus for that PR? I am myself not aware of any problems experienced with channels inside Servo(perhaps with the exception of ipc-channel blocking on sending(see https://github.com/servo/servo/issues/14704). I my experience, Select has been reliable and useful. Were there specific problems that haven't been documented so far?

[SimonSapin]

The fact that using std::sync::mpsc::Select requires unsafe is a very strongly negative point IMO. Having .wait() return an integer that you have to compare to other things yourself also feels error-prone.

My personal motivation for https://github.com/servo/servo/pull/19515 is reducing the number of unstable features used in Servo, especially those that apparently do not have a path to stabilization.

[gterzian]

@SimonSapin I see, those are worthy goals indeed.

So essentially, if Select were to ditch the unsafe add, and improve the wait api, it could be moved into stable? Would there be interest from someone to review this kind of work, a path towards merging it? I have the summer ahead of me, if someone is willing to get some pretty naive implementations thrown at them...

What is the story behind there not being a path towards stabilization? It certainly doesn't seem to be due to a lack of brains in the community writing channel implementations...