users are not added to (all) groups

[dmccuk]

Hi Tomas, I was looking for a way contact you and this was the only one I could find. I'm having a problem getting groups assigned to the users I'm trying to add - I've installed puppet-accounts & gpasswd.

Hiera:

accounts::groups:
  engineers:
    gid: 503
    members: ['user1', 'john']

accounts::users:
  user1:
    comment: "Linux user1"
    groups: ['engineers' ]
    shell: "/bin/bash"
    ssh_keys:
      'DM@here':
        type: "ssh-rsa"
        key: "a_valid_public_ssh_key_string"
  john:
    comment: "John Doe"
    groups: ["sudo", "users"]
    shell: "/bin/bash"
    pwhash: "$6$GDH43O5m$FaJsdjUta1wXcITgKekNGUIfrqxYogW"
    ssh_keys:
      'john@doe': # an unique indentifier of a key
        type: "ssh-rsa"
        key: "AAAAAABBBBBCCCCC"

---

Output on puppet run:

Info: Applying configuration version '1457439312'
Notice: /Stage[main]/Accounts::Groups/Accounts::Group[sudo]/Group[sudo]/ensure: created
Notice: /Stage[main]/Accounts::Groups/Accounts::Group[engineers]/Group[engineers]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[john]/Group[john]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[user1]/User[user1]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[user1]/File[/home/user1/.ssh]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[user1]/File[/home/user1/.ssh/authorized_keys]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[user1]/Ssh_authorized_key[DM@here]/ensure: created
Info: FileBucket got a duplicate file {md5}d41d8cd98f00b204e9800998ecf8427e
Notice: /Stage[main]/Accounts::Users/Accounts::User[john]/User[john]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[john]/File[/home/john/.ssh]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[john]/File[/home/john/.ssh/authorized_keys]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[john]/Ssh_authorized_key[john@doe]/ensure: created
Info: FileBucket got a duplicate file {md5}d41d8cd98f00b204e9800998ecf8427e
Notice: Finished catalog run in 3.60 seconds

[root@home]# groups user1
user1 : user1
[root@home]# groups john
john : john

Could he help me to resolve?

Regards

Dennis

[deric]

Hey, no problem. I should have warned you, this stuff is still a bit experimental :wink:. The log output is from the first puppet run, right? Have you tried if the group's members are modified during second run?

The group management in Puppet is really incomplete, so I've tried moving to gpasswd. Now I realized that it might be better to create users first and then assign them to groups (and not the other way round as it is now - that was due to Puppet resource management).

If it's not created during next runs, try to comment out this line, that's where the magic starts :star2:.

One more thing, which OS do you use? This group management is becoming more low-level thing.

[dmccuk]

Thanks for getting back to me. I appreciate this is new and I'm probably finding an early issue. Does it work for you using my hiera data?

I tried your suggestion to comment out this line and re-ran puppet a few times (after clearing out the groups and users):

#    'attribute_membership' => 'inclusive',

puppet versions: master: 3.8.4 Agent (remote node): 3.6.2

OS: master: CentOS Linux release 7.2.1511 (Core) Agent node: Red Hat Enterprise Linux Server release 6.7 (Santiago)

Output from puppet:

# puppet agent -t
Info: Retrieving plugin
Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/lib/puppet/lib/facter/service_provider.rb
Info: Loading facts in /var/lib/puppet/lib/facter/package_provider.rb
Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb
Info: Caching catalog for server1
Info: Applying configuration version '1457446917'
Notice: /Stage[main]/Accounts::Groups/Accounts::Group[sudo]/Group[sudo]/ensure: created
Notice: /Stage[main]/Accounts::Groups/Accounts::Group[engineers]/Group[engineers]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[user1]/Group[user1]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[john]/Group[john]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[user1]/User[user1]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[john]/User[john]/ensure: created
Notice: /Stage[main]/Accounts::Users/Accounts::User[john]/Ssh_authorized_key[john@doe]/ensure: created
Notice: Finished catalog run in 3.17 seconds
[root@home]# groups user1
user1 : user1
[root@home]# groups john
john : john
[root@home]# puppet agent -t
Info: Retrieving plugin
Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/lib/puppet/lib/facter/service_provider.rb
Info: Loading facts in /var/lib/puppet/lib/facter/package_provider.rb
Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb
Info: Caching catalog for server1
Info: Applying configuration version '1457446945'
Notice: Finished catalog run in 3.13 seconds
[root@home]# puppet agent -t
Info: Retrieving plugin
Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/lib/puppet/lib/facter/service_provider.rb
Info: Loading facts in /var/lib/puppet/lib/facter/package_provider.rb
Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb
Info: Caching catalog for server1
Info: Applying configuration version '1457446956'
Notice: Finished catalog run in 2.69 seconds

[root@home]# groups user1
user1 : user1
[root@home]# groups john
john : john
[root@home]# id user1
uid=500(user1) gid=504(user1) groups=504(user1)
[root@tdi-tst04-fl home]# id john
uid=501(john) gid=505(john) groups=505(john)

Regards

Dennis

[deric]

Ok thanks, I've tested this so far on Debian based systems. The problem might be with the gpasswd command on RedHat. I guess it must exist there, otherwise you would see some error message.

When you add some debugging line here like:

Puppet.info("cmd: #{run_cmd}")

You should see something like gpasswd -a john engineers, so either this command ends with success and nothing happens or this doesn't happen at all due to some other problem.

[dmccuk]

Hi, I check gpasswd and the options seem to be exactly the same in redhat as debian.

I added the line you specified above but didn't see the output you described. so it now looks like this (is this correct?).

  def mod_group(cmds)
    cmds.each do |run_cmd|
      begin
        execute(run_cmd,:custom_environment => @custom_environment)
        Puppet.info("cmd: #{run_cmd}")
      rescue Puppet::ExecutionFailure => e
        if $?.exitstatus == 3 then
          Puppet.warning("Modifying #{@resource[:name]} => #{e}")
        else
          raise e
        end
      end
      Puppet.debug("Success: #{run_cmd}")
    end
  end
end

after resting the users and groups, I ran puppet in debug mode and found this message though (it may not be related):

Debug: /Group[engineers]: Provider groupadd does not support features manages_members; not managing attribute members
Debug: Executing '/usr/sbin/groupadd -g 503 engineers'
Notice: /Stage[main]/Accounts::Groups/Accounts::Group[engineers]/Group[engineers]/ensure: created
Debug: /Group[engineers]: The container Accounts::Group[engineers] will propagate my refresh event
Debug: Accounts::Group[engineers]: The container Class[Accounts::Groups] will propagate my refresh event

Unfortunately, there is no reference to gpasswd in the output so I wonder if I've set it up correctly.

Regards

[deric]

Cool, finally we're getting somewhere :+1: this should be the cause of all troubles:

 Provider groupadd does not support features manages_members; not managing attribute members

What is the output of

facter -y | grep operatingsystem

on client node?

Here we define, that the provider supports manages_members, but it's not understood by Puppet.

[dmccuk]

Hi, here's the output:

# facter -y | grep operatingsystem
  operatingsystem: RedHat
  operatingsystemmajrelease: "6"
  operatingsystemrelease: "6.7"

Regards

[deric]

Ok, that should be just fine. One more thing, could you check if pluginsync is enabled?

puppet config print | grep pluginsync

Can you try running puppet agent -td --pluginsync ?

[dmccuk]

Hi, I ran those but didn't see anything unfortunately

On both the master and puppet client I get the same result for the check:

# puppet config print | grep pluginsync
pluginsync = true

on the client:

# puppet agent -t --pluginsync
Info: Retrieving plugin
Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/lib/puppet/lib/facter/service_provider.rb
Info: Loading facts in /var/lib/puppet/lib/facter/package_provider.rb
Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb
Info: Caching catalog for server1
Info: Applying configuration version '1457528002'
Notice: Finished catalog run in 2.31 seconds

Regards

[deric]

Strange, let's check this:

$ puppet config print libdir
/var/lib/puppet/lib

and in your libdir you should have this file puppet/provider/group/gpasswd.rb:

ls -laF /var/lib/puppet/lib/puppet/provider/group/gpasswd.rb 
-rw-r--r-- 1 root root 3450 Mar  6 17:09 /var/lib/puppet/lib/puppet/provider/group/gpasswd.rb

is that correct?

[dmccuk]

Hi, the libdir is the same:

puppet config print libdir
/var/lib/puppet/lib

But not the gpasswd.rb. I use my own (test) environment but these are the locations:

locate gpasswd.rb
/etc/puppet/environments/production/modules/roles/gpasswd/lib/puppet/provider/group/gpasswd.rb
/home/dennis.mccarthy/puppet/dennismccarthy/modules/accounts/spec/fixtures/modules/gpasswd/lib/puppet/provider/group/gpasswd.rb
/home/dennis.mccarthy/puppet/dennismccarthy/modules/gpasswd/lib/puppet/provider/group/gpasswd.rb

When in installed deric-gpasswd-0.2.0.tar.gz I used the puppet install command to install it (I have no access to the puppetforge repo from the server):

puppet module install deric-gpasswd-0.2.0.tar.gz --ignore-dependencies
Notice: Preparing to install into /etc/puppet/environments/production/modules/roles ...
Notice: Installing -- do not interrupt ...
/etc/puppet/environments/production/modules/roles
└── deric-gpasswd (v0.2.0)

Have I not installed/set it up correctly?

[deric]

Looks like a problem with module path. First of all, check the environment on client:

$ puppet config print environment
production

A quick fix would be reinstalling the module directly into <env>/modules/:

puppet module install --target /etc/puppet/environments/production/modules deric-gpasswd-0.2.0.tar.gz

From the path it looks like you're using the profiles & roles pattern, which is cool. But I'm not really familiar with roles configuration. I guess you would have to include this role into your node definition, something like include gpasswd (but that's just wild guess).

[dmccuk]

Hi, I'm still not able to make it work. I did find this which may not be any help at all but does create the users and the groups:

Class:

class users {
      define add ($group = $name, $ensure, $users) {
        $user_defaults = { gid => $group, ensure => present }
        group { $group: ensure => $ensure }
        create_resources('user', $users, $user_defaults)
      }

      $group_defaults = { ensure => present }
      create_resources('users::add', hiera_hash(groups), $group_defaults)

}

Hiera/common.yaml:

groups:
  engineers:
    users:
      dennis.mccarthy: {}
      user1: {}
  sudo:
    users:
      john:
        groups: [ 'engineers' ]

OUTPUT:

# puppet agent -t
Info: Retrieving plugin
Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/lib/puppet/lib/facter/service_provider.rb
Info: Loading facts in /var/lib/puppet/lib/facter/package_provider.rb
Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb
Info: Caching catalog for tdi-tst04-fl.test.local
Info: Applying configuration version '1457539534'
Notice: /Stage[main]/Users/Users::Add[engineers]/Group[engineers]/ensure: created
Notice: /Stage[main]/Users/Users::Add[engineers]/User[dennis.mccarthy]/ensure: created
Notice: /Stage[main]/Users/Users::Add[engineers]/User[user1]/ensure: created
Notice: /Stage[main]/Users/Users::Add[sudo]/Group[sudo]/ensure: created
Notice: /Stage[main]/Users/Users::Add[sudo]/User[john]/ensure: created
Notice: Finished catalog run in 2.57 seconds

[root@home]# groups user1
user1 : engineers
[root@home]# groups dennis.mccarthy
dennis.mccarthy : engineers
[root@home]# groups john
john : sudo engineers

[deric]

Do you have an environment.conf file in /etc/puppet/environments/production/ ?

I guess if you define something like:

modulepath = modules/roles:modules:$basemodulepath

it should work. See puppet docs.

[noqqe]

Im having the same issue (using SLES). Group memberships via

accounts::groups:
  devs:
    members: ['user1', 'user2']
    gid: 6004

and also this has no effect:

  user1:
    uid: 6014
    groups:
      - 'users'
      - 'devs'

After that, I've read on the issue here and installed gpasswd module. Now (after two runs, pluginsync) it says:

Notice: /Stage[main]/Accounts::Groups/Accounts::Group[users]/Group[users]/members: members changed '' to 'user1'
Notice: /Stage[main]/Accounts::Groups/Accounts::Group[devs]/Group[devs]/members: members changed '' to 'user1,user2'

But after all, there is no change in the group file taken place.

[noqqe]

Looks like this is the problem:

Debug: Executing '/usr/bin/gpasswd -d user1 devs'
Debug: Success: /usr/bin/gpasswd -d user1 devs
Debug: Executing '/usr/bin/gpasswd -a user1 devs'
Debug: Success: /usr/bin/gpasswd -a user1 devs

BUT, when i try to do this manually:

/usr/bin/gpasswd: invalid option -- 'd'
/usr/bin/gpasswd: invalid option -- 'a'

:/

[deric]

Hey @noqqe, thanks for testing this. I was going to close this issue as not related not the module, it seems to me as an installation issue.

Meanwhile I've started working on acceptance tests. It's working on Ubuntu 12.04 (and 14.04). I don't have any box with SLES yet, but I'm gonna look into that. It just took me a while to configure just single test properly. It's the dummiest scenario, we're not removing accounts or groups. It's possible that Puppet resources behave differently between Linux distributions.

Here's a log of the spec run on Ubuntu: https://gist.github.com/deric/ec2e33084398f238848c

[dmccuk]

@noqqe i think if you're using red hat 6 it may not work the same as it does on ubuntu. I got this one working quite quickly using hiera.https://github.com/greenaar/puppet-hiera_users It was quick to set up and has most of the options you need.

Thanks for you help trying to debug Tomas.

[deric]

It looks like the group provider is not supported at all on CentOS:

Debug: Reloading groupadd group provider
Debug: Resource group[engineers] was not determined to be defined
Debug: Create new resource group[engineers] with params attribute_membershipinclusivemembersjohngid158ensurepresent
Debug: Resource group[users] was not determined to be defined
Debug: Create new resource group[users] with params attribute_membershipinclusivemembersjohngid100ensurepresent
Debug: Resource group[john] was not determined to be defined
Debug: Create new resource group[john] with params beforeAnchor[accounts::user::groups::john]gidundefensurepresent

Then the attribute members does not have any effect.

[deric]

Ok, the problem was in the gpasswd module as members of groups were incorrectly reported. Also the tests were also incomplete, because I was testing assignment to single group (so it wasn't really working on Debian). I've successfully run acceptance specs on Debian 7, Ubuntu 14.04 and CentOS 6. Right now the test scenario is fairly simple, but preparing the test boxes was much more time demanding than fixing the module. Well, hopefully next releases will be more stable.

The bug is fixed in v1.3.1.