Closed git-developer closed 6 months ago
Works as expected on master branch without docker. My "production" instance does not have this problem too.
Looks like modbus thread are finished properly but there is a problem with mqtt shutdown. I suspect a race condition between ModMqtt::notifyQueues and ModMqtt::waitForQueues.
Does adding 5 sec wait in ModMqtt::notifyQueues() helps?
I spent some time trying to reproduce this problem without any success. You may try to build debug version and if deadlock occurs then connect debugger and see where it is stuck.
Unfortunately there is a chance that in debug build this will not happen at all :(
Thanks for investigating this issue, I really appreciate that.
It's helpful to know that you can't reproduce the issue in your environment. That could mean that the problem might be introduced by my build and/or runtime environment (Docker). I will try to find out whether the problem is related to some of the container's base components (Linux Alpine / musl).
The problem does not occur when Debian is used as base image instead of Alpine, so the problem is probably not caused by the code of this project.
I replaced while loop with predicate in waitForQueues in 1.4.0. Still no idea what the problem is, so it may or may not fix it :-)
Thanks for releasing 1.4.0, I just tried it. Behavior is unchanged. The last message in the log is
Stopping mosquitto message loop
But only on Alpine (based on musl), no problem on Debian (based on glibc). Seems to be related to the queue lock. I don't think this is something we can fix here.
Could you share Dockerfile for alpine build so I can run it from within source code tree?
Of course, the Dockerfile is from the Docker PR.
Thanks. This is a problem with mosquitto. I reported it as Issue 2981. Before thinking about workarounds I plan to wait for a while and see what will happen with this issue.
Observed behavior
When a
SIGTERM
is sent tomodmqttd
, the process seems to be shutting down properly:But after that, the process keeps running:
A second
pkill
finally terminates the process.Expected behavior
I expect that
modmqttd
is terminated when aSIGTERM
is sent.Steps to reproduce
This behavior can be reproduced with the following command (where
some-demo-config.yaml
contains the config):Within the container:
Remarks
When a
SIGHUP
is sent instead of aSIGTERM
(pkill -HUP modmqttd
), the process terminates immediately with no log output (looks like no cleanup is done then).