search

s3rius / FastAPI-template

Feature rich robust FastAPI template.
MIT License
1.99k stars 173 forks source link

Worker process interrupted #225

Open rcholic opened 1 week ago

rcholic commented 1 week ago

I use taskiq with ListQueueBroker (backed by reddis). In the log, I see a lots of worker process interrupted:

2024-10-23 22:22:38.363 | INFO     | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Listening started.
2024-10-23 22:22:39.445 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Worker process interrupted.
2024-10-23 22:22:39.446 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Shutting down the broker.
2024-10-23 22:22:42.697 | INFO     | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Listening started.
2024-10-23 22:22:43.684 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Worker process interrupted.
2024-10-23 22:22:43.686 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Shutting down the broker.
2024-10-23 22:22:46.688 | INFO     | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Listening started.
2024-10-23 22:22:47.921 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Worker process interrupted.
2024-10-23 22:22:47.923 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Shutting down the broker.
2024-10-23 22:22:50.809 | INFO     | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Listening started.
2024-10-23 22:22:52.167 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Worker process interrupted.
2024-10-23 22:22:52.181 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Shutting down the broker.
2024-10-23 22:22:55.158 | INFO     | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Listening started.
2024-10-23 22:22:56.489 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Worker process interrupted.
2024-10-23 22:22:56.492 | WARNING  | trace_id=0 | span_id=0 | logging:callHandlers:1706 - Shutting down the broker.

Is there a way to keep the worker and broker keep running?

s3rius commented 1 week ago

Can you enable DEBUG logging to see where the error might be?

rcholic commented 1 week ago

Can you enable DEBUG logging to see where the error might be?

hmm, I do have following setting turned on for debug mode in settings, but I don't see errors that lead to the worker interruption issue. Anywhere elese I can enable DEBUG logging? thanks!

log_level: LogLevel = LogLevel.DEBUG

s3rius commented 1 week ago

Can you supply taskiq worker cli with --log-level DEBUG argument?

Like taskiq worker --log-level DEBUG ....

rcholic commented 1 week ago

Can you supply taskiq worker cli with --log-level DEBUG argument?

Like taskiq worker --log-level DEBUG ....

I'm using docker-compose to start the worker like following:

  taskiq-worker:
    <<: *main_app
    labels: []
    command:
    - taskiq
    - worker
    - -fsd
    - my_app.tkq:broker

to add log-level , is following correct?

  taskiq-worker:
    <<: *main_app
    labels: []
    command:
    - taskiq
    - worker
    - -fsd
    - --log-level DEBUG
    - my_app.tkq:broker
s3rius commented 1 week ago

Yes, should be correct.