search

processone / ejabberd

Robust, Ubiquitous and Massively Scalable Messaging Platform (XMPP, MQTT, SIP Server)
https://www.process-one.net/en/ejabberd/
Other
6.03k stars 1.5k forks source link

Ejabberd crashes without generating error logs and crash dumps #4058

Closed member7me closed 1 year ago

member7me commented 1 year ago

Environment

Bug description

Hello. Similar bug to https://github.com/processone/ejabberd/issues/4048 and https://github.com/processone/ejabberd/issues/4054 Ejabberd 21.01 worked fine. After updating Debian to version 12 (bookworm) ejabberd also updated to 23.01. After that service started to crash. Fully randomly. Since 5-60 minutes after starting. Firstly crashes ejabberdctl, i.e. ejabberdctl status stops answering. All this time service still working, web admin also working. ejabberdctl live doesn't start. In few minutes after that full ejabber service crashes. Service ejabberd status gone into inactive status. Webadmin crashes also. Whole service is restarting. All this happens without ending at intervals of 5-60 minutes, randomly. Config file didn't change.

error.log and ejabberd.log are empty, I mean - only with standart warning messages. No errors. At all. Also there is no CPU or RAM high usage.

Only downgrading to the old version Ejabberd 21.01 solves the problem.

member7me commented 1 year ago

Screenshot from other, but similar server with similar configs. It's test server, where ejabberd wasn't updated, but installed as fresh from Debian 12 distro. journalctl -u ejabberd.service 1

example of rebootings 2

licaon-kter commented 1 year ago

Did you follow the steps in https://docs.ejabberd.im/admin/upgrade/ after the upgrade? Eg. If using SQL etc

member7me commented 1 year ago

Did you follow the steps in https://docs.ejabberd.im/admin/upgrade/ after the upgrade? Eg. If using SQL etc

I don't use sql, just mnesia.

prefiks commented 1 year ago

You probably can fix this temporary by removing WatchdogSec=30 from ejabberd.service, looks like for some reason systemd doesn't get "pings" from ejabberd.

member7me commented 1 year ago

I noticed, this bug exists only when ejabberd is under load with active connected members on board. If to upload same environment on other server where there is no active connected members, there will be no errors and service crash.