containerd / ttrpc-rust

Rust implementation of ttrpc (GRPC for low-memory environments)
Apache License 2.0
195 stars 45 forks source link

[release/0.4] sync/server: close conn fd in the reaper thread #104

Closed fuweid closed 2 years ago

fuweid commented 2 years ago

The sync/server uses accepted fd as key to maintain the connection and closes the fd before client_handler thread exits. It will cause the deadlock between listen and reaper threads. The timeline is like:

reaper          listener            client-handler-1

                            close(fd=100)

            fd(=100)=accepted

            start client_handler-2

            locked key-values
            update (100, client_handler-2)
            unlocked

                            notify reaper fd=100

received fd=100
locked key-values
wait-client_handler-2
                            exit(0)

            fd(=101)=accepted
            acquiring lock

The lowest fd will be returned by accepted. The listener thread will use new connection handler to replace the existing one. Since there is long-running connection for container lifecycle, the reaper thread holds the lock which listener thread is acquiring. It is deadlock.

To fix this issue, we should close fd in the reaper thread to prevent the reuse fd.

Signed-off-by: Wei Fu fuweid89@gmail.com (cherry picked from commit db3639c2669ceb29eee83c2954a97855f7243abb) Signed-off-by: Wei Fu fuweid89@gmail.com

Tim-Zhang commented 2 years ago

The pr https://github.com/containerd/ttrpc-rust/pull/105 fixed the CI problems, please wait for it's merging.

fuweid commented 2 years ago

@Tim-Zhang Thanks!

fuweid commented 2 years ago

@Tim-Zhang thanks for quick reply. Is it possible to make new release for crate after merge?

Tim-Zhang commented 2 years ago

@fuweid The version 0.4.16 has been released https://crates.io/crates/ttrpc/0.4.16