search

resgateio / resgate

A Realtime API Gateway used with NATS to build REST, real time, and RPC APIs, where all your clients are synchronized seamlessly.
https://resgate.io
MIT License
685 stars 67 forks source link

Server not stopping with correct signals #35

Closed ghost closed 5 years ago

ghost commented 6 years ago

I setup resgate to run in tandem with some other processes as part of my setup.

Using go-cmd, and raised an issue here:

https://github.com/go-cmd/cmd/issues/19#issuecomment-406827169

The test code to reproduce I don't have accessible but the 20 line example code in go-cmd is the same and so very easy to try.

It is important that spawned servers inside NATS terminate correctly in order for them to be well managed by the process manager. I have not tried resgate with systemd or launchd or windows service yet. I intend to try that next

jirenius commented 6 years ago

Great that you try these things. Thanks!

Stopping of resgate can be initiated in two ways:
By signal (os.Interrupt, os.Kill, syscall.SIGHUP, syscall.SIGTERM, or syscall.SIGQUIT)
Or by resgate internally deciding to stop, which happens if:

In any of these cases, resgate will try to do a graceful shutdown for 10 seconds, after which it will exit with a panic (to provide a stacktrace as of why it failed to shut down):
https://github.com/jirenius/resgate/blob/master/main.go#L202

That means, resgate should always exit within 10 seconds after receiving a signal.

But it might exit without calling os,Exit, or it might exit with a panic.

What sort of termination is required by go-cmd (or any of the other wrappers, for that matter)?

I will try to reproduce the issue as well.

ghost commented 6 years ago

The last question about what sort of termination.. I don't know to be frank. The source for go-cmd is pretty lean so maybe worth looking at.

Or maybe solve the problem. The only reason I raised this is to enable Devs and end users to easily add Microservice on the same box or a set of boxes and just have everything work and die and log to one place. In other words frictionless deployment with docker or K8 !

Hashicorp nomad is a pretty good option that I got working running resgate,. Go-res and node from a single command. It updates the whole deployment whenever the source is committed.

So it sort off comes down to deciding if nomad is the approach or something like go-cmd. I favour nomad, consul, vault. Together they give the big boy enterprise stuff everyone needs without needing docker or K8. If you want to use docker and K8 nomad supports that too btw.

I can finish my setup and upload it. But we should decide this because they are tied to other issues. Like security and if vault is used.

Or we can leave it alone too and just use small bits of code to accomplish deployment and updates.

Curious what your own objectives here are.

On Tue, 24 Jul 2018, 09:40 Samuel Jirénius, notifications@github.com wrote:

Great that you try these things. Thanks!

Stopping of resgate can be initiated in two ways: By signal https://github.com/jirenius/resgate/blob/master/main.go#L179 (os.Interrupt, os.Kill, syscall.SIGHUP, syscall.SIGTERM, or syscall.SIGQUIT) Or by resgate internally deciding to stop https://github.com/jirenius/resgate/blob/master/main.go#L188, which happens if:

In any of these cases, resgate will try to do a graceful shutdown for 10 seconds, after which it will exit with a panic (to provide a stacktrace as of why it failed to shut down): https://github.com/jirenius/resgate/blob/master/main.go#L202

That means, resgate should always exit within 10 seconds after receiving a signal.

But it might exit without calling os,Exit, or it might exit with a panic.

What sort of termination is required by go-cmd (or any of the other wrappers, for that matter)?

I will try to reproduce the issue as well.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jirenius/resgate/issues/35#issuecomment-407312104, or mute the thread https://github.com/notifications/unsubscribe-auth/ATuCwi-mzJhuz-vnzmj5A-G5nR6yCkYQks5uJs-IgaJpZM4VZ4WD .

jirenius commented 6 years ago

I assume we are talking about solving the problem of deploying/starting/stopping a set of micro services, that together composes a server application, to ease up the deployment and development process?

I haven't tried nomad, consul, or vault, but have rather just used docker as container for a group of micro services. Or for smaller projects, with limited number of services (1 - 3), the micro services are running as separate processes.

I don't really have any objectives myself, rather than to enable using resgate+microservices with the technology of choice, be it go-cmd, docker, K8, or HashiCorps solutions. And for this, if the resgates fails to send the proper error codes, preventing a solution to be used, it needs to be fixed!

But from what I understand, nomad is more used for creating service-to-service communication. RES protocol concerns service-to-client, and while RES-service protocol can also be used for service-to-service communcation, it doesn't have to. Both can exists in parallel: nomad with gRPC for faster service-to-service communcation, and RES for the client, with all the realtime data, security and caching that it provides.

jirenius commented 5 years ago

I've investigated the exit codes returned by Resgate.

Resgate does respond to the Interrupt signal, as expected, exiting with code 0 on successful shutdown. In other cases, such as disconnects from NATS server, Resgate correctly exits with error code 1.

I’m closing this issue because it has been inactive for a few months, and without being able to reproduce the issue.