c-mart commented 7 years ago

ISSUE TYPE

Bug Report

COMPONENT NAME

with_items, host groups

ANSIBLE VERSION

ansible 2.2.0.0

CONFIGURATION

https://github.com/cyverse/openstack-ansible-host-prep/blob/master/ansible/ansible.cfg

OS / ENVIRONMENT

Ubuntu 16.04 LTS

SUMMARY

My organization uses a role that generates a list of root passwords, writes them to a file, encrypts them, and sets them on target hosts. It works with Ansible 2.1.2.0 but not with Ansible 2.2.0.0 which was just released to the Ubuntu PPA two days ago.

I think the problem starts with this task:

- name: generate root password
  local_action: shell pwgen {{ TARGET_HOST_PASSWORDS.password_length }} 1
  with_items: groups['target-hosts']
  when: GENERATE_ROOT_PASSWORDS is defined
  run_once: true
  register: root_pass

It's supposed to generate a password corresponding to each host in a group and store it in a root_pass variable, so we can later encrypt the passwords and set them on each of the target hosts. The intention is that root_pass.results is a list; each member item in the list is a dictionary that contains a stdout of the generated password, and an item showing the name of the host it is generated for.

However! In the above task, it looks like with_items: groups['target_hosts'] isn't actually expanded out to a hostname for each item. If I debug out root_pass.results...

- name: see root_pass
  debug:
    var: "{{ item }}"
  with_items: "{{ root_pass.results }}"

...then I see output like this (repeated for each of the members of target-hosts):

TASK [host-credentials : see passwords] ****************************************
ok: [host01] => (item={'_ansible_parsed': True, '_ansible_item_result': True, u'end': u'2016-11-03 17:13:43.692185', '
_ansible_no_log': False, u'stdout': u'eiGeigho6shi', u'cmd': u'pwgen 12 1', u'changed': True, 'invocation': {'module_
name': u'command', u'module_args': {u'creates': None, u'executable': None, u'_uses_shell': True, u'_raw_params': u'pw
gen 12 1', u'removes': None, u'warn': True, u'chdir': None}}, 'item': u"groups['target-hosts']", u'delta': u'0:00:00.
002931', u'stderr': u'', u'rc': 0, '_ansible_delegated_vars': {'ansible_host': u'localhost'}, 'stdout_lines': [u'eiGe
igho6shi'], u'start': u'2016-11-03 17:13:43.689254', u'warnings': []}) => {
    "<type 'dict'>": "VARIABLE IS NOT DEFINED!", 
    "item": {
        "changed": true, 
        "cmd": "pwgen 12 1", 
        "delta": "0:00:00.002931", 
        "end": "2016-11-03 17:13:43.692185", 
        "invocation": {
            "module_args": {
                "_raw_params": "pwgen 12 1", 
                "_uses_shell": true, 
                "chdir": null, 
                "creates": null, 
                "executable": null, 
                "removes": null, 
                "warn": true
            }, 
            "module_name": "command"
        }, 
        "item": "groups['target-hosts']", 
        "rc": 0, 
        "rc": 0, 
        "start": "2016-11-03 17:13:43.689254", 
        "stderr": "", 
        "stdout": "eiGeigho6shi", 
        "stdout_lines": [
            "eiGeigho6shi"
        ], 
        "warnings": []
    }
}

root_pass.results.(item).item is assigned a string/unicode value of groups['target-hosts'] for each member of root_pass.results, rather than the actual name of a host in the target-hosts group.

That is weird because I can do the following, and I get the expected results, with groups['target-hosts'] expanded out to a host for each item:

- name: see target-hosts
  debug:
    var: "{{ item }}"
  with_items: groups['target-hosts']

TASK [host-credentials : see target-hosts] *************************************
ok: [host01] => (item=groups['target-hosts']) => {
    "groups['target-hosts']": [
        "host01", 
        "host02", 
        "host03", 
        "host04", 
        "host05", 
        "host06"
    ], 
    "item": "groups['target-hosts']"
}

Anyhow, the failure to expand groups['target-hosts'] in the above local_action task seems to break subsequent tasks, which need a way to determine hostnames and passwords:

TASK [host-credentials : output password to file] ******************************
fatal: [host01]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a
 variable that is undefined. The error was: 'unicode object' has no attribute 'item'\n\nThe error appears to have bee
n in '/root/openstack-ansible-host-prep/ansible/roles/host-credentials/tasks/main.yml': line 62, column 3, but may\nb
e elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: outp
ut password to file\n  ^ here\n"}

If I go back in time and install the Ansible 2.1.2.0-1 .DEB from the PPA, then the role completes without error! So, the behavior is definitely new with 2.2.

If we are doing this a stupid way and should refactor, feel free to say so!

STEPS TO REPRODUCE

Run this playbook with real hosts configured in the target-hosts group (like this).

EXPECTED RESULTS

The host-credentials role/playbook completes without error

ACTUAL RESULTS

TASK [host-credentials : output password to file] ******************************
fatal: [host01]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a
 variable that is undefined. The error was: 'unicode object' has no attribute 'item'\n\nThe error appears to have bee
n in '/root/openstack-ansible-host-prep/ansible/roles/host-credentials/tasks/main.yml': line 62, column 3, but may\nb
e elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: outp
ut password to file\n  ^ here\n"}

ansibot commented 7 years ago

@ansible ping, this issue is waiting for your response. click here for bot help

scjody commented 7 years ago

I can reproduce this. Here's the simplest playbook I can come up with:

---
- hosts: tag_environment_stage:&tag_roles_backstage
  gather_facts: False
  remote_user: ubuntu
  sudo: True
  tasks:
  - name: restart all the nodewebkits
    monit: name={{ item.name }} state=restarted
    with_items: node_webkits

Here's the error:

fatal: [ec2-54-163-173-183.compute-1.amazonaws.com]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'unicode object' has no attribute 'name'\n\nThe error appears to have been in '/mnt/nfshome/plotly/streambed/deployment/playbook_stage.yml': line 7, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  tasks:\n  - name: restart all the nodewebkits\n    ^ here\n"}

As above, this worked in 2.1.x but is broken in 2.2.x.

The interesting part is if I remove the optional "name" from my task I get:

fatal: [ec2-54-163-173-183.compute-1.amazonaws.com]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'unicode object' has no attribute 'name'\n\nThe error appears to have been in '/mnt/nfshome/plotly/streambed/deployment/playbook_stage.yml': line 7, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  tasks:\n  - monit: name={{ item.name }} state=restarted\n    ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes.  Always quote template expression brackets when they\nstart a value. For instance:\n\n    with_items:\n      - {{ foo }}\n\nShould be written as:\n\n    with_items:\n      - \"{{ foo }}\"\n"}

Unfortunately, quoting {{ item.name }} does not fix the issue.

To be clear, here is the playbook used to produce the second error:

---
- hosts: tag_environment_stage:&tag_roles_backstage
  gather_facts: False
  remote_user: ubuntu
  sudo: True
  tasks:
  - monit: name={{ item.name }} state=restarted
    with_items: node_webkits

bcoca commented 7 years ago

Possible Misunderstanding

Hi!

Thanks very much for your submission to Ansible. It sincerely means a lot to us.

We believe the ticket you have filed is being somewhat misunderstood, as one thing works a little differently than stated.

You should have been getting deprecation warnings with those plays in 2.0/2.1 as 'bare' variables in with_ were scheduled for removal, you need to change your plays to:

  with_items: "{{groups['target-hosts']}}"

or

 with_items: "{{node_webkits}}"

This was done so we can distinguish between 'undefined' and 'simple strings' in loop items.

In the future, this might be a topic more well suited for the user list, which you can also post here if you'd like some more help with the above.

https://groups.google.com/forum/#!forum/ansible-project

Thank you once again for this and your interest in Ansible!