[post] Graceful shutdown a http server from its handler in Rust warp

at15 commented 4 years ago

Type

request post from @at15

Description

NOTE: The up to date note is in http://doc/github.com/at15/rust-learning/lib/warp/

I want to have a /shutdown route that allows me to shutdown a http sever by hitting that endpoint. However it is not that easy in Rust compared with Go (create a context/channel for cancellation, start two go routine, one blocks on server, one wait for cancel). Rust code is harder because it uses ownership instead of GC and has mutability rules.

The post will contain the following

how to do graceful shutdown in Go
the iterations of getting the shutdown logic working in Rust
- closure trait Fn, FnMut, FnOnce
- channel (in tokio)
- passing a channel to Fn closure
- mutex

Update

xplorld commented 4 years ago

typo in the URL

at15 commented 4 years ago

@xplorld no it's by design, it's a private repo and I have a g3doc like server running locally for showing markdown files.

xplorld commented 4 years ago

cool

On Sun, Jul 5, 2020, 12:54 AM Pinglei Guo notifications@github.com wrote:

@xplorld https://github.com/xplorld no it's by design, it's a private repo and I have a g3doc like server running locally for showing markdown files.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dyweb/blog/issues/51#issuecomment-653854795, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABVSYTT6TQJ7NSTOJPMS5M3R2AWTXANCNFSM4OQV3Q3Q .

xplorld commented 4 years ago

Not a Rustacean, but can we have a global bool, read-write lock protected (leaning read), and /shutdown simply places the bool to false so that next turn the listener would understand the situation and quit?

at15 commented 4 years ago

From library user perspective,I think the crate does not expose a interface that allow you to customize the behavior for each new tcp connection/http request. Even if it does, this is an extra overhead that somehow 'synchronize' all the connections because they need to lock and check a flag that is true for most of the time, a rw lock is still a lock and may not be cheaper than a simple mutex.

(I think) The common approach is to notify the server to graceful shutdown (w/ a timeout) so it can flush out in flight requests as much as possible. It should not wait forever for a buggy handler logic/buggy client. In go you starts a new goroutine that waits on the shutdown signal from a channel, that go routine also contains a reference to the server, the server runs in another go routine.

wg.Add(2)
go func() {
  server.Listen()
  wg.Done()
}
go func() {
  <-shutdown // blocks until signal
  server.shutdown()
  wg.Done()
}
wg.Wait()

From library author side, I think the implementation can be similar to what you said, once the library user signals shutdown, a flag inside the server is set so it rejects incoming connection/request and exit current thread(s)/goroutine(s) on timeout.

And for my rust problem ... I found the main cause is I misunderstood how closure trait FnOnce is determined and think all the struct methods are using borrow instead of move.

// pseudo code

struct Ry {
   shibai: i64
}

impl Ry {
   fn patpat(&self) {
     println!("patpat shibai de ry {}", self.shibai);
   }

   fn cece(&mut self) {
      self.shibai += 1;
     println!("cece cg, ry shibai {}", self.shibai);
  }

  fn gg(self) {
     println!("protobuf consumed ry");
  }
}

fn main() {
    let r1 = Ry{shibai: 1};
    let r2 = Ry{shibai: 2};
    let r3 = Ry{shibai: 3};
    let c1 = || {
       r1.patpat();  // Fn because patpat is immutable borrow
   }
   let c2 = || {
      r2.cece(); // FnMut because cece is mutable borrow
  }
  let c3 = || {
     r3.gg(); // FnOnce because gg takes ownership of the ry instance, there is no `move` keyword for the closure, but the content in the closure determined the closure uses move instead of borrow
  }
}

xplorld commented 4 years ago

I'm not sure about it, but tokio::runtime::Runtime::shutdown_on_idle may help.

Pinglei Guo notifications@github.com 于2020年7月5日周日下午9:05写道：

From library user perspective,I think the crate does not expose a interface that allow you to customize the behavior for each new tcp connection/http request. Even if it does, this is an extra overhead that somehow 'synchronize' all the connections because they need to lock and check a flag that is true for most of the time, a rw lock is still a lock and may not be cheaper than a simple mutex.

(I think) The common approach is to notify the server to graceful shutdown (w/ a timeout) so it can flush out in flight requests as much as possible. It should not wait forever for a buggy handler logic/buggy client. In go you starts a new goroutine that waits on the shutdown signal from a channel, that go routine also contains a reference to the server, the server runs in another go routine.

wg.Add(2)go func() { server.Listen() wg.Done() }go func() { <-shutdown // blocks until signal server.shutdown() wg.Done() }wg.Wait()

From library author side, I think the implementation can be similar to what you said, once the library user signals shutdown, a flag inside the server is set so it rejects incoming connection/request and exit current thread(s)/goroutine(s) on timeout.

And for my rust problem ... I found the main cause is I misunderstood how closure trait FnOnce is determined and think all the struct methods are using borrow instead of move.

// pseudo code

struct Ry { shibai: i64 }

impl Ry { fn patpat(&self) { println!("patpat shibai de ry {}", self.shibai); }

fn cece(&mut self) { self.shibai += 1; println!("cece cg, ry shibai {}", self.shibai); }

fn gg(self) { println!("protobuf consumed ry"); } }

fn main() { let r1 = Ry{shibai: 1}; let r2 = Ry{shibai: 2}; let r3 = Ry{shibai: 3}; let c1 = || { r1.patpat(); // Fn because patpat is immutable borrow } let c2 = || { r2.cece(); // FnMut because cece is mutable borrow } let c3 = || { r3.gg(); // FnOnce because gg takes ownership of the ry instance, there is no move keyword for the closure, but the content in the closure determined the closure uses move instead of borrow } }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dyweb/blog/issues/51#issuecomment-654003545, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABVSYTUNZWJ5MVFRB55KI33R2FESFANCNFSM4OQV3Q3Q .

at15 commented 4 years ago

I got it working now and thanks @xplorld for following along in the thread

wrap the sender using Option, the take method is take(&mut self) -> Option<T> so you get owned value from mutable reference
change mutable to immutable requires using Mutex
because the handler also requires Clone trait, Arc<Mutex<Option<Sender>>>

Still need some time to clean up the note and write small examples (in rust playground) so ppl familiar w/o go but w/ little rust background can also understand it (e.g. @gaocegege)

dyweb / blog