unmanaged known host entries not actually being removed

[ninneman]

We (me and my organization) have saz-ssh installed and running in Puppet 6.15. It's set to the default behavior of loading collected host keys into /etc/ssh/ssh_known_hosts, and to purge any unmanaged entries. When Puppet runs, the log shows lots of lines like this: Notice: /Stage[main]/Ssh/Sshkey[host.example.com]/ensure: removed (corrective)

We've taken this to mean that saz-ssh is correctly noting spurious entries and removing them. But they don't actually disappear! This is happening on every single Puppet run on every single Puppet-managed node. As nodes go offline or change state, the volume of these Notice lines is growing.

Any advice would be appreciated. We've been troubleshooting this for days, and can't come up with any explanation for why this would be happening.

[saz]

Are you setting resources { 'sshkey': purge => true } with any other value for purge within your manifests?

If storeconfigs_enabled => true and purge_unmanaged_sshkeys => true, this module sets resources { 'sshkey': purge => true } which is working for me as expected.

Is this happening on all managed nodes or are there exceptions?

[ninneman]

This is happening on every single host where saz-ssh is in use; we checked.

Our manifests/Hiera don't set any purge-related values at all. Should they? If so, what? Neither of these are documented (the word "purge" doesn't occur in the readme here or on the Puppet Forge page).

[saz]

The module is managing that, as long as you're not disabling it (it's enabled by default).

Which version of this module are you using? Which distribution and version? Can you show me the part of your manifest related to this module?

[ninneman]

Thanks. I thought so based on a casual code dive, but wasn't sure.

We're using 6.1.0; the latest release.

We have a number of different OSs being managed:

• CentOS 6.10
• CentOS 7.8.2003
• RedHat 6.10
• RedHat 7.7
• RedHat 7.8
• Ubuntu 16.04
• Ubuntu 18.04

Here's our manifest:

include ::ssh

Then we have some Hiera to configure details:

ssh::server_options:
  PrintMotd: 'no'
  Banner: '/etc/issue'
  AuthorizedKeysCommand: '/usr/bin/sss_ssh_authorizedkeys'
  GSSAPIAuthentication: 'yes'
  UsePAM: 'yes'

[ninneman]

I realized we have puppetlabs-sshkeys_core installed. It also is invoked by include ssh or similar. Could this be colliding with saz-ssh in weird ways?

[saz]

Have you found a solution for your issue?

[ninneman]

It could be a problem with the Puppet master and/or PuppetDB. We're in the process of standing up a new set of these. I'll report here on whether that takes care of it or not.

[saz]

@ninneman are there any news on this issue?

[saz]

As there isn't any feedback, I'll close this issue. Feel free to reopen it, if it's still an issue for you