The sensu-agent is unable to start due to "bucket already exists", and we have the following output from journalctl
Jan 23 15:27:44 test systemd[1]: Starting The Sensu Agent process....
Jan 23 15:27:44 test systemd[1]: Started The Sensu Agent process..
Jan 23 15:27:44 test sensu-agent[22032]: {"component":"agent","level":"info","msg":"compacting api queue","time":"2024-01-23T15:27:44-05:00"}
Jan 23 15:27:44 test sensu-agent[22032]: {"component":"agent","level":"info","msg":"finished api queue compaction","time":"2024-01-23T15:27:44-05:00"}
Jan 23 15:27:44 test sensu-agent[22032]: {"component":"agent","error":"error creating agent: error compacting queue: bucket already exists","level":"fatal","msg":"error executing sensu-agent","time":"2024-01-23T15:27:44-05:00"}
Jan 23 15:27:44 test systemd[1]: sensu-agent.service: main process exited, code=exited, status=1/FAILURE
Jan 23 15:27:44 test systemd[1]: Unit sensu-agent.service entered failed state.
Jan 23 15:27:44 test systemd[1]: sensu-agent.service holdoff time over, scheduling restart.
Jan 23 15:27:44 test systemd[1]: Stopping The Sensu Agent process....
Jan 23 15:27:44 test systemd[1]: Starting The Sensu Agent process....
Jan 23 15:27:44 test systemd[1]: sensu-agent.service start request repeated too quickly, refusing to start.
Jan 23 15:27:44 test systemd[1]: Failed to start The Sensu Agent process..
Jan 23 15:27:44 test systemd[1]: Unit sensu-agent.service entered failed state.
Possible Solution
I know for a fact that removing the file /var/cache/sensu/sensu-agent/.lasr.temp.db makes sensu-agent to start successfully.
Steps to Reproduce (for bugs)
Unfortunately, I'm unable to reproduce the issue. However, we've already seen this issue happen a few times in production environments.
I've included the .last.temp.db file in case it helps:
lasr.temp.zip
And also strace when trying to execute the sensu-agent in foreground:
strace-lasr.txt
Your Environment
# sensu-agent version is 6.10.0 (compiled from source)
sensu-agent version (devel)+ce, community edition, built with go1.18.3
# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 8 (jessie)"
NAME="Debian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=debian
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
Expected Behavior
Current Behavior
The sensu-agent is unable to start due to "bucket already exists", and we have the following output from journalctl
Possible Solution
I know for a fact that removing the file
/var/cache/sensu/sensu-agent/.lasr.temp.dbmakes sensu-agent to start successfully.Steps to Reproduce (for bugs)
Unfortunately, I'm unable to reproduce the issue. However, we've already seen this issue happen a few times in production environments.
I've included the
.last.temp.dbfile in case it helps: lasr.temp.zipContext
I've capture the following inotify events:
And also strace when trying to execute the sensu-agent in foreground: strace-lasr.txt
Your Environment