gajus / lightship

Abstracts readiness, liveness and startup checks and graceful shutdown of Node.js services running in Kubernetes.
Other
517 stars 33 forks source link

`Stop` does not properly wait for the server to close #5

Closed dcherman closed 5 years ago

dcherman commented 5 years ago

Thanks for providing this lib, I was about to write something like this when I saw you mention this in the k8s slack channel :)

Was looking through the code and noticed that the stop method does not properly wait for the server to close since no callback is provided. Since the method was marked async, it seems like you had intended for it to wait.

https://github.com/gajus/lightship/blob/c45367275ddccdd1d2df4f4db9839f0f3e3e5822/src/factories/createLightship.js#L133

gajus commented 5 years ago

Thank you

gajus commented 5 years ago

:tada: This issue has been resolved in version 2.0.1 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

dcherman commented 5 years ago

Hey @gajus, thanks for taking a look at this. If you don't mind a suggestion, I'd create a new promise that resolves when your server is closed (or fails to close I guess) rather than calling process.exit() since you can't assume that other resources that the application uses like databases, redis, etc.. have all been cleaned up yet

gajus commented 5 years ago

This behaviour is by design.

The idea is that Lightship registers all the shutdownHandler and forces termination after all shutdown handlers have been called.

I have added a warning about the active event loop preventing the program to gracefully exit and changed the exit code to 1.

Does this seem reasonable solution?

If not, please advise how would you handle it, since the requirement is to terminate the program within a given time limit.

gajus commented 5 years ago

:tada: This issue has been resolved in version 2.1.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

dcherman commented 5 years ago

Ah, yea, that makes a lot of sense. I was thinking about this incorrectly. I think the warning is a nice touch though since that might help people find dangling resources that they missed cleaning up.

In that case, my only remaining suggestion would be to use Promise.all and map to return an array of promises from the shutdown handlers. The current behavior would call each of them in series and possibly cause shutdown to take longer than it otherwise should. Don't think we necessarily need to enforce order here since that can be accomplished in userland if necessary.

gajus commented 5 years ago

In that case, my only remaining suggestion would be to use Promise.all and map [..]

Cannot really do that.

Suppose that user does this.

const connection = [..];

registerShutdownHandler(async () => {
  await connection.query([..]);
});

registerShutdownHandler(async () => {
  await connection.end();
});

Using Promise.all would potentially interrupt whatever the dependent shutdown handler is doing.

dcherman commented 5 years ago

Sure, but do you feel that it's correct for users to depend on the ordering of shutdown handlers in your lib? It feels more correct for them to have done something like:

  registerShutdownHandler(async () => {
    try {
      await connection.query([..]);
    } finally {
      await connection.end();
    }
});

rather than depending on implementation details of this lib to determine whether or not the resource is still available

gajus commented 5 years ago

Fair point.

I will come back to this in a bit.

https://github.com/gajus/lightship/issues/6

Don't want to rush the change. Will check how comparable libraries implement shutdown handler execution.

gajus commented 4 years ago

This wouldn't be safe though:

registerShutdownHandler(async () => {
    try {
      await connection.query([..]);
    } finally {
      await connection.end();
    }
});

registerShutdownHandler(async () => {
    try {
      await connection.query([..]);
    } finally {
      await connection.end();
    }
});
gajus commented 4 years ago

Just do:

registerShutdownHandler(async () => {
  await Promise.all([
    userCollectionOfShutdownHandlers,
  ]);
});

if you need parallelism.