fornellas / resonance

A transactional host configuration management tool.
GNU General Public License v3.0
3 stars 1 forks source link

Fix agent shutdown race condition #30

Open fornellas opened 2 months ago

fornellas commented 2 months ago

The current code here is quite hacky:

func PostShutdownFn(ctx context.Context) func(http.ResponseWriter, *http.Request) {
    return func(w http.ResponseWriter, r *http.Request) {
        go func() {
            time.Sleep(1 * time.Second)
            os.Exit(0)
        }()
    }
}

When a shutdown request is made, it sleeps for 1s and then exit, in the hopes that the POST request will have finished being served within this time period.

Go's HTTP server does provide a proper interface for graceful shutdown, which we should use. I believe that, once the SSH connection is closed, the connection pipe that's connected to the agent stdin / stdout will be closed, and the agent process should receive a SIGPIPE, which if we register the correct signal handlers to implement graceful shutdown, should be a cleaner solution. The shutdown post could just reuse this, and send a SIGTERM to self.

fornellas commented 1 month ago

For reference: after me + @ricardo6142dh chatted, we agreed on the conclusion that we're better off to move to gRPC. So let's do it.