containerd / ttrpc-rust

Rust implementation of ttrpc (GRPC for low-memory environments)
Apache License 2.0
195 stars 45 forks source link

sync/server: close conn fd in the reaper thread #101

Closed fuweid closed 2 years ago

fuweid commented 2 years ago

The sync/server uses accepted fd as key to maintain the connection and closes the fd before client_handler thread exits. It will cause the deadlock between listen and reaper threads. The timeline is like:

reaper          listener            client-handler-1

                            close(fd=100)

            fd(=100)=accepted

            start client_handler-2

            locked key-values
            update (100, client_handler-2)
            unlocked

                            notify reaper fd=100

received fd=100
locked key-values
wait-client_handler-2
                            exit(0)

            fd(=101)=accepted
            acquiring lock

The lowest fd will be returned by accepted. The listener thread will use new connection handler to replace the existing one. Since there is long-running connection for container lifecycle, the reaper thread holds the lock which listener thread is acquiring. It is deadlock.

To fix this issue, we should close fd in the reaper thread to prevent the reuse fd.

Signed-off-by: Wei Fu fuweid89@gmail.com

fuweid commented 2 years ago

ping @Tim-Zhang @lifupan @liubin @teawater ~

wllenyj commented 2 years ago

LGTM

Tim-Zhang commented 2 years ago

@lifupan PTAL

fuweid commented 2 years ago

thanks! @Tim-Zhang @lifupan @mxpv is it possible to release crate for this bug fix?