Open TJDev opened 1 year ago
@TJDev i recommend to check logdna-agent service log:
journalctl -u logdna-agent
@dkhokhlov There are no entries in the log before I have started it manually
I assume the service is enabled:
systemctl list-unit-files | grep logdna-agent
May be start condition is not working. Unit file says to start after network.target:
$ cat logdna-agent.service
[Unit]
Description=Logdna Agent
Documentation=https://docs.logdna.com
After=network.target
is it started?
systemctl | grep network.target
to see whole journal log since last boot:
journalctl -b
I assume the service is enabled:
systemctl list-unit-files | grep logdna-agent
There is no entry for logdna-agent.
is it started?
systemctl | grep network.target
Network is started
What happens if you try to enable the service and test it again?
systemctl enable logdna-agent
sh-4.2$ sudo systemctl list-unit-files | grep logdna-agent
sh-4.2$ sudo systemctl enable logdna-agent
logdna-agent.service is not a native service, redirecting to /sbin/chkconfig.
Executing /sbin/chkconfig logdna-agent on
sh-4.2$ sudo systemctl list-unit-files | grep logdna-agent
sh-4.2$ sudo systemctl status logdna-agent
● logdna-agent.service - LSB: Starts LogDNA Agent at boot
Loaded: loaded (/etc/rc.d/init.d/logdna-agent; bad; vendor preset: disabled)
Active: inactive (dead)
Docs: man:systemd-sysv-generator(8)
sh-4.2$ sudo systemctl start logdna-agent
sh-4.2$ sudo systemctl status logdna-agent
● logdna-agent.service - LSB: Starts LogDNA Agent at boot
Loaded: loaded (/etc/rc.d/init.d/logdna-agent; bad; vendor preset: disabled)
Active: active (exited) since Mon 2022-12-05 09:36:07 UTC; 2s ago
Docs: man:systemd-sysv-generator(8)
Process: 14144 ExecStart=/etc/rc.d/init.d/logdna-agent start (code=exited, status=0/SUCCESS)
Dec 05 09:36:07 systemd[1]: Starting LSB: Starts LogDNA Agent at boot...
Dec 05 09:36:07 logdna-agent[14144]: Already started
Dec 05 09:36:07 systemd[1]: Started LSB: Starts LogDNA Agent at boot.
my bad, "old" agent, Sys-V based service. the log goes to:
/var/log/logdna-agent.log
@TJDev resolved?
@dkhokhlov Sorry for the late response. Unfortunately the issue is still open. The agent does not start after deployment.
What is in the log?
/var/log/logdna-agent.log
2023-03-30 12:24:50 0000 [info] logdna-agent/2.2.1 started on ELB-env (x.x.x.x) 2023-03-30 12:24:50 0000 [info] LogDNA URL: https://logs.logdna.com:443/logs/agent 2023-03-30 12:24:50 0000 [info] streaming /var/log: 25 file(s) 2023-03-30 12:25:50 0000 [info] streaming /var/log: 6 new file(s), 31 total file(s) 2023-03-30 12:25:54 0000 [info] got SIGTERM signal, shutting down... 2023-03-30 12:42:57 0000 [info] logdna-agent/2.2.1 started on ELB-env (x.x.x.x) 2023-03-30 12:42:57 0000 [info] LogDNA URL: https://logs.logdna.com:443/logs/agent 2023-03-30 12:42:57 0000 [info] streaming /var/log: 32 file(s)
Deployment on 12:23, manual restart on 12:42
there was agent shutdown - SIGTERM, graceful stop request before you started it again. that is the reason agent was not running when you were looking. I would check system logs. runlevel changes. also check runlevel of agent service.
I expect the SIGTERM
has a relation to the shutdown of cfn-hub as they happen at the same time.
sh-4.2$ sudo cat /var/log/messages | grep 08:13
Apr 21 08:13:36 ip-172-31-37-214 systemd: Stopping This is cfn-hup daemon...
Apr 21 08:13:43 ip-172-31-37-214 dhclient[3120]: XMT: Solicit on eth0, interval 111720ms.
Apr 21 08:13:46 ip-172-31-37-214 systemd: Stopped This is cfn-hup daemon.
2023-04-21 08:13:36 0000 [info] got SIGTERM signal, shutting down...
cfn-hub is about AWS config changes...
lets temporary try to disable agent stop in init script to isolate - comment this whole section out in /etc/init.d/logdna-agent
:
expected:
With whole section you mean the complete if-else-branch, line 76 to 105?
lets keep top if and echo and remove this: https://github.com/logdna/logdna-agent/blob/f0d219d91c08ac0f7c9299a99e2b9ef9ac6d707d/tools/files/linux/init-script#L78-L102
I will switch to logdna-agent-v2 as I have also out of memory exceptions for time to time and the systemd unit file should restart the agent in that case.
I'm running an app on AWS Elastic Beanstalk running Amazon Linux 2 with Docker platform. I have changed deployment to immutable, so that with every app update a new EC2 instance is spawned as I currently have only one instance running and do want minimal downtime. Unfortunately the LogDNA Agent does not start up and after every deployment I have to start it manually (
sudo systemctl start logdna-agent
).I had used the script v1 and also tried to use services key or platform hooks but nothing worked.
Is there anything I can do or debug why the service is not starting?