Closed 3c2b2ff5 closed 2 years ago
What is the Linux distribution involved? Do you have commercial support?
I am running Debian 10 with 5.10.0-0.bpo.3-amd64 Kernel. I am afraid we don't have a commercial support!
Does the problem occur when you execute systemctl daemon-reload
manually, outside a Puppet context? If it does I think you would get better answers by writing to a Debian mailing list or forum.
It happens when systemctl daemon-reload
is executed within puppet context
@3c2b2ff5 do you have an example puppet report/output that we can see which resources are involved?
The only real connection between systemd and snmpd i identfied at a quick look is https://github.com/voxpupuli/puppet-snmp/blob/master/manifests/init.pp#L460-L463 or if snmpd config changes.
It would also be helpful if you can tell us your used versions of the modules (and/or update them to the lastest once if not already done)
I modified /etc/systemd/system/consul.service.d/consul.conf
file and executed systemctl daemon-reload
manually, nothing happened, then I run the puppet agent to restore the config and in syslog
you can see the restart of snmpd
:
2021-07-21T15:06:02.077892+00:00 localhost puppet-agent[11582]: (/Stage[main]/consul/Systemd::Dropin_file[consul.conf]/File[/etc/systemd/system/consul.service.d/consul.conf]) Scheduling refresh of Class[Systemd::Systemctl::Daemon_reload]
2021-07-21T15:06:02.312899+00:00 localhost systemd[1]: Stopping Consul Agent...
2021-07-21T15:06:06.926292+00:00 localhost systemd[1]: consul.service: Succeeded.
2021-07-21T15:06:06.926936+00:00 localhost systemd[1]: Stopped Consul Agent.
2021-07-21T15:06:06.929790+00:00 localhost systemd[1]: Starting Consul Agent...
2021-07-21T15:06:07.019185+00:00 localhost systemd[1]: Started Consul Agent.
2021-07-21T15:06:07.025687+00:00 localhost puppet-agent[11582]: (/Stage[main]/Consul::Run_service/Service[consul]) Triggered 'refresh' from 1 event
2021-07-21T15:06:07.026135+00:00 localhost puppet-agent[11582]: (Class[Systemd::Systemctl::Daemon_reload]) Scheduling refresh of Exec[systemctl-daemon-reload]
2021-07-21T15:06:07.036345+00:00 localhost systemd[1]: Reloading.
2021-07-21T15:06:07.188868+00:00 localhost puppet-agent[11582]: (/Stage[main]/Systemd::Systemctl::Daemon_reload/Exec[systemctl-daemon-reload]) Triggered 'refresh' from 1 event
2021-07-21T15:06:07.189873+00:00 localhost puppet-agent[11582]: (Class[Systemd::Systemctl::Daemon_reload]) Scheduling refresh of Service[snmpd]
2021-07-21T15:06:07.247316+00:00 localhost systemd[1]: Stopping Simple Network Management Protocol (SNMP) Daemon....
2021-07-21T15:06:07.248033+00:00 localhost snmpd[16564]: Received TERM or STOP signal... shutting down...
2021-07-21T15:06:07.249842+00:00 localhost Keepalived_vrrp[25863]: AgentX master disconnected us, reconnecting in 15
2021-07-21T15:06:07.251901+00:00 localhost systemd[1]: snmpd.service: Succeeded.
2021-07-21T15:06:07.252706+00:00 localhost systemd[1]: Stopped Simple Network Management Protocol (SNMP) Daemon..
2021-07-21T15:06:07.254869+00:00 localhost systemd[1]: Starting Simple Network Management Protocol (SNMP) Daemon....
2021-07-21T15:06:07.257770+00:00 localhost systemd[1]: Started Simple Network Management Protocol (SNMP) Daemon..
2021-07-21T15:06:07.262014+00:00 localhost puppet-agent[11582]: (/Stage[main]/Snmp/Service[snmpd]) Triggered 'refresh' from 1 event
2021-07-21T15:06:07.282456+00:00 localhost snmpd[12815]: Turning on AgentX master support.
2021-07-21T15:06:07.283718+00:00 localhost snmpd[12815]: Turning on AgentX master support.
2021-07-21T15:06:07.285059+00:00 localhost snmpd[12815]: NET-SNMP version 5.7.3
2021-07-21T15:06:14.498210+00:00 localhost puppet-agent[11582]: Applied catalog in 30.74 seconds
Well I can update some modules at the next schedule, we need to coordinate this first. But for now the module versions are:
mod 'KyleAnderson/consul', '7.0.2'
mod 'puppet/snmp', '5.1.1'
mod 'puppet/keepalived', '3.1.0'
You're at the latest version of this module so I think you'll have to get the module modified for your use case, or submit a PR.
Thank you very much @ubellavance
You're at the latest version of this module so I think you'll have to get the module modified for your use case, or submit a PR.
How can I get it modified?
You're at the latest version of this module so I think you'll have to get the module modified for your use case, or submit a PR.
How can I get it modified?
You can wait until someone does, or try to find a puppet consultant that will do the PR for you.
OK, sure thanks
This an other incident I had yesterday:
2021-07-25T06:49:52.476020+00:00 localhost systemd[1]: Starting Daily apt upgrade and clean activities...
2021-07-25T06:49:54.718776+00:00 localhost systemd[1]: apt-daily-upgrade.service: Succeeded.
2021-07-25T06:49:54.719219+00:00 localhost systemd[1]: Started Daily apt upgrade and clean activities.
2021-07-25T06:50:01.103590+00:00 localhost CRON[6530]: (root) CMD ( ipset-save > /etc/ipset)
2021-07-25T06:50:01.104716+00:00 localhost CRON[6532]: (root) CMD ( /usr/local/sbin/iptables-save > /etc/iptables/rules.v4)
2021-07-25T06:51:47.296931+00:00 localhost Keepalived_vrrp[3099]: AgentX master disconnected us, reconnecting in 15
2021-07-25T06:51:47.297133+00:00 localhost Keepalived_vrrp[3099]: AgentX master agent failed to respond to ping. Attempting to re-register.
2021-07-25T06:51:47.297191+00:00 localhost Keepalived_vrrp[3099]: scheduler: Error performing epoll_ctl DEL op for fd:15 (Bad file descriptor)
2021-07-25T06:52:02.297994+00:00 localhost Keepalived_vrrp[3099]: NET-SNMP version 5.7.3 AgentX subagent connected
I believe this is now fixed with 6.0.0. I hope so at least.
Hi,
I am seeing a strange behavior of snmpd. When ever
systemctl daemon-reload
is triggered, snmpd is restarted.I am running keepalived with snmp support to be able to monitor the state of my vrrp instances, but when snmpd is restarted, keepalived needs a restart as well, otherwise the connection to AgentX is lost.
How is this connected to systemd? could it be that the agent socket is managed by systemd? How can I avoid this behavior?
Thank you very much