rust-lang / wg-async

Working group dedicated to improving the foundations of Async I/O in Rust
https://rust-lang.github.io/wg-async/
Apache License 2.0
379 stars 88 forks source link

[meta] blog post quest issue #61

Open nikomatsakis opened 3 years ago

nikomatsakis commented 3 years ago

This issue exists to collect links to blog posts that seem like status quo story ideas. Please provide the link and whatever other details you can that might help people to know what it's about.

This issue is not an end point! It's more of a "work queue". The idea is for folks to read the blog posts and identify potential status quo user stories and then open up fresh issues with those stories.

Please check the box next to a blog post link (and leave a comment) if you have done that!

Suggested format for each comment:

* [ ] BLOG_POST_URL

> Some quote from blog post that gives the general idea
nikomatsakis commented 3 years ago

As many of the readers already know, for the past (almost) 2 years, I've been developing a Rust-crate for D-Bus, called zbus. With this being my first big Rust project to start from scratch done almost entirely in my spare time, the progress was rather slow for many months. My perfectionism didn't help much with the progress either but also the fact that implementing a Serde API for the D-Bus format was quite challenging. Then along came my old friend and ex-colleague, Marc-André Lureau who sped up the progress 10 times and soon after we had the 1.0 release of zbus.

While my original plan (perfectionism again) was for the API to be primarily async, with the synchronous API mostly just a wrapper around it, it was Marc-Andre who ended up doing most of the work and coming up with nice high-level API and his use case was primarily synchronous so we decided to go with synchronous API first. I still believe that was the right thing to do, since neither of us were familiar with async programming in Rust and going with the original plan would have meant the first release getting delayed by at least another half an year.

This may sound very disappointing to readers who come from glib programming background but a purely synchronous blocking API in a Rust app is not at all as bad it would be in a glib+C (or even Vala) app. There is a reason why Rust is famous for its fearless concurrency.

nikomatsakis commented 3 years ago

In 2013, I discovered the Rust programming language and quickly decided to learn it and make it my main programming language.

In 2017, I moved to Berlin and joined Parity as a Rust developer. The task that occupied my first few months was to build rust-libp2p, a peer-to-peer library in asynchronous Rust (~89k lines of code at the moment). Afterwards, I integrated it in Substrate (~400k lines of code), and have since then been the maintainer of the networking part of the code base.

In the light of this recent blog post and this twitter interaction, I thought it could be a good idea to lay down some of the issues I’ve encountered over time through experience.

Please note that I’m not writing this article on behalf of my employer Parity. This is my personal feedback when working on the most asynchronous-heavy parts of Parity-related projects, but I didn’t show it to anyone before publishing and it might differ from what the other developers in the company would have to say.

nikomatsakis commented 3 years ago

Most of the web services I've written in Rust have used actix-web. Recently, I needed to write something that will provide some reverse proxy functionality. I'm more familiar with the hyper-powered HTTP client libraries (reqwest in particular). I decided this would be a good time to experiment again with hyper on the server side as well. The theory was that having matching Request and Response types between the client and server would work nicely. And it certainly did.

In the process, I ended up with an interesting example of battling ownership through closures and async blocks. This is a topic I typically mention in my Rust training sessions as the hardest thing I had to learn when learning Rust. So I figure a blog post demonstrating one of these crazy cases would be worthwhile.

nikomatsakis commented 3 years ago

Recently I have been getting into async and tokio. With great success despite the tribulations I brought up here: A year in Rust. And I still can't read the docs! 6

My latest creation has tokio-tunstentite serving up web sockets to a test client also using tokio-tungstenite.

Having got that basic ws functionality working I proceeded to introduce NATS messaging into the mix using the nats crate. You see, those web socket clients are expecting to be fed data that derives from subscription to our NATS server (among other things). Each ws client's data requirements may require multiple subscriptions to different 'subjects' from NATS.

So here comes the problem:

nikomatsakis commented 3 years ago

Ever since it’s illegal for me to leave my house, my weekends have been filled with rewriting Whisperfish. Whisperfish is an app, originally by Andrew E. Bruno, that natively implements Signal for SailfishOS. My goal with the rewrite is to modernize the non-GUI code such that it uses the official libsignal-protocol-c instead of the Go-reimplementation. For this, I would either use C++ or Rust; the title of the post probably spoiled which one I prefer.

I’m imagining two target audiences for this blog post: either you’re a Rustacean, and you’re here for the Tokio and Actix magic, or (and that’s not xor) you’re from the SailfishOS community and you’re wondering what all the Tokio and Actix buzzwords are even about. With that in mind, I’ll make an introduction on both topics, and depending on your background, you can skip either.

magnet commented 3 years ago
nikomatsakis commented 3 years ago

In this post we will explore a brief example of asynchronous programming in Rust with the Tokio runtime, demonstrating different execution scenarios. This post is aimed at beginners to asynchronous programming.

nikomatsakis commented 3 years ago

In this post we will set up a very simple data ingestion process with Rust and AWS Lambda.

nikomatsakis commented 3 years ago
Lesson I — The Rust language does not include any intrinsic high-level means to poll futures. That is left as an “exercise for the reader”

Lesson II — It is possible for a single dependency of your crate to be so tightly coupled to a future polling runtime that it effectively makes that runtime mandatory for all consumers.

Lesson III — If you drop (or allow to be dropped) the tokio runtime, any pending tasks are cancelled.

Lesson IV — Each runtime has a single blocking entry point, so it can only ever block on a single unit of asynchronous work.
nikomatsakis commented 3 years ago

This reddit thread has a lot of good material!

ibraheemdev commented 3 years ago
nikomatsakis commented 3 years ago

thanks to some excellent git bisecting from ceejbot, and a nudge in the right direction from tomaka17’s post “a look back at async rust” (in particular, the “flow control is hard” section), we were able to diagnose a surprising production rustlang crossbeam channel deadlock.

https://twitter.com/isntitvacant/status/1375861944322727938

nikomatsakis commented 3 years ago

An excellent article from @fasterthanline covering pinning, threads, async tasks.

ibraheemdev commented 3 years ago