zeromq / goczmq

goczmq is a golang wrapper for CZMQ.
Mozilla Public License 2.0
591 stars 94 forks source link

Problem: actor errors block channeler shutdown #288

Closed tarmo closed 4 years ago

tarmo commented 4 years ago

Solution: keep goroutine running to allow graceful shutdown

Currently if any error is returned by ZMQ calls made inside the Channeler actor goroutine it triggers an immediate exit of the that goroutine. This makes it impossible to shut down the companion channeler goroutine (and allow it to free its allocated resources) because the only path for doing so is sending "destroy" to commandChan which then causes the channeler goroutine to block forever waiting for a response to the "destroy" command from a now missing actor goroutine here: https://github.com/zeromq/goczmq/blob/4dc887d7239b715e613e2937d5aaf58d2033c8b0/channeler.go#L205

The proposed solution keeps the actor goroutine working and hopefully able to receive and process the "destroy" command even after returning an error. Presuming whatever error it received doesn't also prevent that from happening.

I've tested the change manually using pub/sub scripts from https://github.com/zeromq/goczmq/pull/285 and by adding randomly returned errors to the relevant methods as I'm unaware of another way to make those functions return an error on demand.

To illustrate the issue, without the fix, if an error is returned from poller.Wait and the application then calls Destroy it results in this goroutine staying around:

github.com/zeromq/goczmq/v4._C2func_zframe_recv(0x7fc0e000e4a0, 0x0, 0x0, 0x0)
    _cgo_gotypes.go:154 +0x56
github.com/zeromq/goczmq/v4.(*Sock).RecvFrame.func1(0xc000112040, 0xc00019e008, 0xc000090dc0, 0x4be386)
    /home/tarmo/goczmq/sock.go:319 +0x55
github.com/zeromq/goczmq/v4.(*Sock).RecvFrame(0xc000112040, 0xc000000000, 0x5128e0, 0x59eff8, 0x7, 0x7fc0e000e4a0, 0xc000198010)
    /home/tarmo/goczmq/sock.go:319 +0x5d
github.com/zeromq/goczmq/v4.(*Sock).RecvMessage(0xc000112040, 0xc000198010, 0x7, 0x7, 0x0, 0x0)
    /home/tarmo/goczmq/sock.go:375 +0x52
github.com/zeromq/goczmq/v4.(*Channeler).channeler(0xc0000d2000, 0xc00002e120, 0xc00002e180)
    /home/tarmo/goczmq/channeler.go:205 +0x61e
created by github.com/zeromq/goczmq/v4.newChanneler
    /home/tarmo/goczmq/channeler.go:265 +0x2e5