Duplicate Bonsai Assets when using multiple namespaces

tdmike commented 4 years ago

Description of problem

When defining the same Bonsai Assets for multiple namespaces you get a duplicate declaration

What did you do? Created two namespaces to enable separation between environments, tried to add the same check to both namespaces

What happened? Received the following error:

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Sensu_bonsai_asset[sensu-plugins/sensu-plugins-cpu-checks] is already declared at (file: /etc/puppetlabs/code/environments/development/site-modules/profile/manifests/sensu_manage.pp, line: 30); cannot redeclare (file: /etc/puppetlabs/code/environments/development/site-modules/profile/manifests/sensu_manage.pp, line: 35) (file: /etc/puppetlabs/code/environments/development/site-modules/profile/manifests/sensu_manage.pp, line: 35, column: 3) on node sensu01.dev.mgt.osp.its.uu.nl

What did you expect to happen? I expected this to happen, due to the usage of the asset name in de declaration.
How can someone reproduce the problem? Create two namespaces Add same check to both namespaces

Command used and debugging output

Is this a masterless or master based puppet setup? This is a master bases puppet setup

Platform and version information

Your OS: Red Hat Enterprise Linux 7.
Your Ruby version:
Your version of Puppet: Puppet Enterprise 5.5.10
Your version of Sensu: 5.16.1
Your version of this module: 4.4.0

ghoneycutt commented 4 years ago

@tdmike Could you please share with us the code you are using for the two conflicting resources.

tdmike commented 4 years ago

Sure:

      sensu_bonsai_asset { 'sensu-plugins/sensu-plugins-cpu-checks' :
        ensure    => 'present',
        version   => 'latest',
        namespace => 'development',
      }
      sensu_bonsai_asset { 'sensu-plugins/sensu-plugins-cpu-checks' :
        ensure    => 'present',
        version   => '4.0.0',
        namespace => 'production',
      }

ghoneycutt commented 4 years ago

In Puppet you cannot have two resources with the same type and title, so that will cause the duplicate declaration. One work around for this is to change the title, such as 'sensu-plugins/sensu-plugins-cpu-checks in development'.

Check out https://github.com/sensu/sensu-puppet#composite-names-for-namespaces and let us know if this helps.

tdmike commented 4 years ago

So basically the same way as in a check. I'll try that out. You might want to clarify that in the docs. (Now that I read it, it's clear to me, but an example regarding sensu_bonsai_asset might help a little bit)

treydock commented 4 years ago

All the Puppet types that support a namespace will also support composite names. I'll the docs to emphasize this feature for all supported types.

treydock commented 4 years ago

Reference docs have been updated with composite name examples:

http://sensu.github.io/sensu-puppet/puppet_types/sensu_bonsai_asset.html

tdmike commented 4 years ago

Sorry to comment here, but this doesn't work, at least not how I would expect it to work. @treydock

puppet agent -t
Info: Using configured environment 'its_feature_sensu_agent'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for sensu01.dev.mgt.osp.its.uu.nl
Error: /Stage[main]/Profile::Sensu_manage/Sensu_bonsai_asset[sensu-plugins/sensu-plugins-cpu-checks in ospmgtdev]: Failed to generate additional resources using 'generate': sensu_asset name invalid
Error: /Stage[main]/Profile::Sensu_manage/Sensu_bonsai_asset[sensu-plugins/sensu-plugins-cpu-checks in default]: Failed to generate additional resources using 'generate': sensu_asset name invalid
Error: /Stage[main]/Profile::Sensu_manage/Sensu_bonsai_asset[sensu/sensu-ruby-runtime in ospmgtdev]: Failed to generate additional resources using 'generate': sensu_asset name invalid
Error: /Stage[main]/Profile::Sensu_manage/Sensu_bonsai_asset[sensu-plugins/sensu-plugins-disk-checks in ospmgtdev]: Failed to generate additional resources using 'generate': sensu_asset name invalid
Error: /Stage[main]/Profile::Sensu_manage/Sensu_bonsai_asset[sensu-plugins/sensu-plugins-filesystem-checks in ospmgtdev]: Failed to generate additional resources using 'generate': sensu_asset name invalid
Info: Applying configuration version 'lp0191-its_feature_sensu_agent-4db8752a8ee'
Error: asset add sensu-plugins/sensu-plugins-cpu-checks --rename sensu-plugins/sensu-plugins-cpu-checks in ospmgtdev --namespace ospmgtdev failed
Error message: Execution of '/bin/sensuctl asset add sensu-plugins/sensu-plugins-cpu-checks --rename sensu-plugins/sensu-plugins-cpu-checks in ospmgtdev --namespace ospmgtdev' returned 1: no version specified, using latest: 4.1.0
fetching bonsai asset: sensu-plugins/sensu-plugins-cpu-checks:4.1.0
Error: error putting resource 0 (/api/core/v2/namespaces/ospmgtdev/assets/sensu-plugins%2Fsensu-plugins-cpu-checks%20in%20ospmgtdev): invalid argument(s) received
Error: /Stage[main]/Profile::Sensu_manage/Sensu_bonsai_asset[sensu-plugins/sensu-plugins-cpu-checks in ospmgtdev]/ensure: change from 'absent' to 'present' failed: asset add sensu-plugins/sensu-plugins-cpu-checks --rename sensu-plugins/sensu-plugins-cpu-checks in ospmgtdev --namespace ospmgtdev failed
Error message: Execution of '/bin/sensuctl asset add sensu-plugins/sensu-plugins-cpu-checks --rename sensu-plugins/sensu-plugins-cpu-checks in ospmgtdev --namespace ospmgtdev' returned 1: no version specified, using latest: 4.1.0
fetching bonsai asset: sensu-plugins/sensu-plugins-cpu-checks:4.1.0profile::sensu_manage::sensu_assets:
  sensu-plugins/sensu-plugins-cpu-checks:
    ospmgtdev:
      version: 'latest'
    default:
      version: '4.0.0'
  sensu/sensu-ruby-runtime:
    ospmgtdev:
      version: 'latest'
  sensu-plugins/sensu-plugins-disk-checks:
    ospmgtdev:
      version: 'latest'
  sensu-plugins/sensu-plugins-filesystem-checks:
    ospmgtdev:
      version: 'latest'
Error: error putting resource 0 (/api/core/v2/namespaces/ospmgtdev/assets/sensu-plugins%2Fsensu-plugins-cpu-checks%20in%20ospmgtdev): invalid argument(s) received
Error: asset add sensu/sensu-ruby-runtime --rename sensu/sensu-ruby-runtime in ospmgtdev --namespace ospmgtdev failed
Error message: Execution of '/bin/sensuctl asset add sensu/sensu-ruby-runtime --rename sensu/sensu-ruby-runtime in ospmgtdev --namespace ospmgtdev' returned 1: no version specified, using latest: 0.0.10
fetching bonsai asset: sensu/sensu-ruby-runtime:0.0.10
Error: error putting resource 0 (/api/core/v2/namespaces/ospmgtdev/assets/sensu%2Fsensu-ruby-runtime%20in%20ospmgtdev): invalid argument(s) received
Error: /Stage[main]/Profile::Sensu_manage/Sensu_bonsai_asset[sensu/sensu-ruby-runtime in ospmgtdev]/ensure: change from 'absent' to 'present' failed: asset add sensu/sensu-ruby-runtime --rename sensu/sensu-ruby-runtime in ospmgtdev --namespace ospmgtdev failed
Error message: Execution of '/bin/sensuctl asset add sensu/sensu-ruby-runtime --rename sensu/sensu-ruby-runtime in ospmgtdev --namespace ospmgtdev' returned 1: no version specified, using latest: 0.0.10
fetching bonsai asset: sensu/sensu-ruby-runtime:0.0.10
Error: error putting resource 0 (/api/core/v2/namespaces/ospmgtdev/assets/sensu%2Fsensu-ruby-runtime%20in%20ospmgtdev): invalid argument(s) received
Error: asset add sensu-plugins/sensu-plugins-disk-checks --rename sensu-plugins/sensu-plugins-disk-checks in ospmgtdev --namespace ospmgtdev failed
Error message: Execution of '/bin/sensuctl asset add sensu-plugins/sensu-plugins-disk-checks --rename sensu-plugins/sensu-plugins-disk-checks in ospmgtdev --namespace ospmgtdev' returned 1: no version specified, using latest: 5.0.1
fetching bonsai asset: sensu-plugins/sensu-plugins-disk-checks:5.0.1
Error: error putting resource 0 (/api/core/v2/namespaces/ospmgtdev/assets/sensu-plugins%2Fsensu-plugins-disk-checks%20in%20ospmgtdev): invalid argument(s) received
Error: /Stage[main]/Profile::Sensu_manage/Sensu_bonsai_asset[sensu-plugins/sensu-plugins-disk-checks in ospmgtdev]/ensure: change from 'absent' to 'present' failed: asset add sensu-plugins/sensu-plugins-disk-checks --rename sensu-plugins/sensu-plugins-disk-checks in ospmgtdev --namespace ospmgtdev failed
Error message: Execution of '/bin/sensuctl asset add sensu-plugins/sensu-plugins-disk-checks --rename sensu-plugins/sensu-plugins-disk-checks in ospmgtdev --namespace ospmgtdev' returned 1: no version specified, using latest: 5.0.1
fetching bonsai asset: sensu-plugins/sensu-plugins-disk-checks:5.0.1
Error: error putting resource 0 (/api/core/v2/namespaces/ospmgtdev/assets/sensu-plugins%2Fsensu-plugins-disk-checks%20in%20ospmgtdev): invalid argument(s) received
Error: asset add sensu-plugins/sensu-plugins-filesystem-checks --rename sensu-plugins/sensu-plugins-filesystem-checks in ospmgtdev --namespace ospmgtdev failed
Error message: Execution of '/bin/sensuctl asset add sensu-plugins/sensu-plugins-filesystem-checks --rename sensu-plugins/sensu-plugins-filesystem-checks in ospmgtdev --namespace ospmgtdev' returned 1: no version specified, using latest: 2.0.0
fetching bonsai asset: sensu-plugins/sensu-plugins-filesystem-checks:2.0.0
Error: error putting resource 0 (/api/core/v2/namespaces/ospmgtdev/assets/sensu-plugins%2Fsensu-plugins-filesystem-checks%20in%20ospmgtdev): invalid argument(s) received
Error: /Stage[main]/Profile::Sensu_manage/Sensu_bonsai_asset[sensu-plugins/sensu-plugins-filesystem-checks in ospmgtdev]/ensure: change from 'absent' to 'present' failed: asset add sensu-plugins/sensu-plugins-filesystem-checks --rename sensu-plugins/sensu-plugins-filesystem-checks in ospmgtdev --namespace ospmgtdev failed
Error message: Execution of '/bin/sensuctl asset add sensu-plugins/sensu-plugins-filesystem-checks --rename sensu-plugins/sensu-plugins-filesystem-checks in ospmgtdev --namespace ospmgtdev' returned 1: no version specified, using latest: 2.0.0
fetching bonsai asset: sensu-plugins/sensu-plugins-filesystem-checks:2.0.0
Error: error putting resource 0 (/api/core/v2/namespaces/ospmgtdev/assets/sensu-plugins%2Fsensu-plugins-filesystem-checks%20in%20ospmgtdev): invalid argument(s) received
Error: asset add sensu-plugins/sensu-plugins-cpu-checks:4.0.0 --rename sensu-plugins/sensu-plugins-cpu-checks in default --namespace default failed
Error message: Execution of '/bin/sensuctl asset add sensu-plugins/sensu-plugins-cpu-checks:4.0.0 --rename sensu-plugins/sensu-plugins-cpu-checks in default --namespace default' returned 1: fetching bonsai asset: sensu-plugins/sensu-plugins-cpu-checks:4.0.0
Error: error putting resource 0 (/api/core/v2/namespaces/default/assets/sensu-plugins%2Fsensu-plugins-cpu-checks%20in%20default): invalid argument(s) received
Error: /Stage[main]/Profile::Sensu_manage/Sensu_bonsai_asset[sensu-plugins/sensu-plugins-cpu-checks in default]/ensure: change from 'absent' to 'present' failed: asset add sensu-plugins/sensu-plugins-cpu-checks:4.0.0 --rename sensu-plugins/sensu-plugins-cpu-checks in default --namespace default failed
Error message: Execution of '/bin/sensuctl asset add sensu-plugins/sensu-plugins-cpu-checks:4.0.0 --rename sensu-plugins/sensu-plugins-cpu-checks in default --namespace default' returned 1: fetching bonsai asset: sensu-plugins/sensu-plugins-cpu-checks:4.0.0
Error: error putting resource 0 (/api/core/v2/namespaces/default/assets/sensu-plugins%2Fsensu-plugins-cpu-checks%20in%20default): invalid argument(s) received
Notice: Applied catalog in 88.53 seconds

Relevant code:

  $sensu_assets.each | String $asset, Hash $asset_hash |{
    $asset_hash.each | String $asset_namespace, Hash $asset_namespace_hash | {
      sensu_bonsai_asset { "${asset} in ${asset_namespace}":
        ensure  => present,
        version => $asset_namespace_hash['version'],
      }
    }
  }

Hiera:

profile::sensu_manage::sensu_assets:
  sensu-plugins/sensu-plugins-cpu-checks:
    ospmgtdev:
      version: 'latest'
    default:
      version: '4.0.0'
  sensu/sensu-ruby-runtime:
    ospmgtdev:
      version: 'latest'
  sensu-plugins/sensu-plugins-disk-checks:
    ospmgtdev:
      version: 'latest'
  sensu-plugins/sensu-plugins-filesystem-checks:
    ospmgtdev:
      version: 'latest'

When only using one namespace, and not using a composite name, this work fine. But with a composite name, it throws a bunch of errors

treydock commented 4 years ago

I'll take a look and see if I can reproduce on my end and come up with a solution.

treydock commented 4 years ago

@tdmike A fix is in #1215. Let me know if this fixes your issues. I applied your example, replacing Hiera with manifest hash and was able to reproduce and actually found 2 bugs.

tdmike commented 4 years ago

@treydock I can confirm this fixes the issue.

sensu / sensu-puppet