search

coinbase / temporal-ruby

Ruby SDK for Temporal
Apache License 2.0
225 stars 84 forks source link

Worker stops polling the temporal server cluster #280

Open Arun-V00222 opened 8 months ago

Arun-V00222 commented 8 months ago

We are using temporal version 0.2.0 in production and are observing the below issue

Container - ECS Instance - EC2

On startup the temporal client is able to poll for activities & workflows available to process and after sometime the polling stops. But the process is still running.

Screenshot 2024-01-12 at 4 43 12 PM

So everytime the process has to be restarted for client to start polling again.

hughevans commented 8 months ago

Are you able to inspect the process? We’re experiencing a similar issue and found that the whole Ruby process for the worker is in a very odd state, not even responding to kill -9.

Arun-V00222 commented 8 months ago

but we are able to kill the process by logging into the container. Supervisor automatically restarts the process and the polling works fine then.

DeRauk commented 8 months ago

Thanks for reporting the issue. When you kill the process is there a stack trace printed out?