odpi / egeria

Egeria core
https://egeria-project.org
Apache License 2.0
786 stars 258 forks source link

Add shutdown hook #6728

Open planetf1 opened 2 years ago

planetf1 commented 2 years ago

Is there an existing issue for this?

Please describe the new behavior that that will improve Egeria

Add a shutdown hook for Egeria, specifically to handle SIGTERM

In operational environments the system/container/process running an Egeria platform may need to be shutdown externally - a system restart, or Kubernetes killing a pod.

These systems are not aware of Egeria APIs, and will typically send a SIGTERM to the jvm, wait about 30s, and then send a SIGKILL (which cannot be handled/blocked)

Egeria should register a shutdown hook, and on received of a SIGTERM or similar, should do what we can to clean down sufficiently cleanly, for a temporary stop. For example this might include attempting to commit the appropriate Kafka offsets, but also it gives opportunity to do other things, for example it may be possible to call connectors to stop - however timescales are short, and we need to take care with any blocking calls.

This may improve resilience/restart time.

Currently the SIGTERM will directly kill the vm - which does of course work

Alternatives

Currently the SIGTERM will directly kill the vm - which does of course work. We may decide this is sufficient

Any Further Information?

No response

Would you be prepared to be assigned this issue to work on?

planetf1 commented 1 year ago

image

Came across these options which may be part of the picture -- this will give the spring application 30s to shut down.

It may be we just do this in demos/charts, but would also want to test/document & understand the shutdown behaviour

planetf1 commented 1 year ago

Adding container label, as in a dynamic k8s environment, scaling operations may mean this is more important, to ensure our integrity

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.