st2 ansible.playbook action user issue

[mab27]

• Running the action with become_user=mab --> OK.

  
  mab@mab-infra:~$ sudo st2 run ansible.playbook playbook=/home/mab/automation/ansible/junos_template/pb.bgp.2.yml cwd=/home/mab/automation/ansible become=true become_user=mab
  ....
  id: 5914a3f37cae2209793f8dc2
  status: succeeded
  parameters: 
  become: true
  become_user: mab
  cwd: /home/mab/automation/ansible
  playbook: /home/mab/automation/ansible/junos_template/pb.bgp.2.yml
  result: 
  failed: false
  return_code: 0
  stderr: ''
  stdout: "
  PLAY [Create BGP junos configuration] ******************************************

TASK [Render BGP configuration for junos devices] ** ok: [vmx1] ok: [vmx2]

TASK [Push bgp configuration on devices] *** ok: [vmx2] ok: [vmx1]

PLAY [Wait for peers to establish connections] *****

TASK [pause] *** Pausing for 1 seconds (ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort) ok: [localhost]

PLAY [Check bgp states] ****

TASK [Check bgp peers states] ** ok: [vmx1] => (item={u'interface': u'ge-0/0/1', u'asn': 102, u'peer_ip': u'172.16.0.40'}) ok: [vmx2] => (item={u'interface': u'ge-0/0/1', u'asn': 101, u'peer_ip': u'172.16.0.30'})

PLAY RECAP ***** localhost : ok=1 changed=0 unreachable=0 failed=0
vmx1 : ok=3 changed=0 unreachable=0 failed=0
vmx2 : ok=3 changed=0 unreachable=0 failed=0
" succeeded: true

- Running the action with become_user=stanley --> KO.

mab@mab-infra:~$ sudo st2 run ansible.playbook playbook=/home/mab/automation/ansible/junos_template/pb.bgp.2.yml cwd=/home/mab/automation/ansible become=true become_user=stanley . id: 5914aa007cae2209793f8de0 status: failed parameters: become: true become_user: stanley cwd: /home/mab/automation/ansible playbook: /home/mab/automation/ansible/junos_template/pb.bgp.2.yml result: failed: true return_code: 2 stderr: Executed command "/opt/stackstorm/virtualenvs/ansible/bin/ansible-playbook --become-user=stanley --become /home/mab/automation/ansible/junos_template/pb.bgp.2.yml" stdout: " PLAY [Create BGP junos configuration] **

TASK [Render BGP configuration for junos devices] ** ok: [vmx1] ok: [vmx2]

TASK [Push bgp configuration on devices] *** fatal: [vmx2]: FAILED! => {"changed": false, "failed": true, "msg": "junos-eznc >= 1.2.2 is required but does not appear to be installed. It can be installed using pip install junos-eznc"} fatal: [vmx1]: FAILED! => {"changed": false, "failed": true, "msg": "junos-eznc >= 1.2.2 is required but does not appear to be installed. It can be installed using pip install junos-eznc"} to retry, use: --limit @/home/mab/automation/ansible/junos_template/pb.bgp.2.retry

PLAY RECAP ***** vmx1 : ok=1 changed=0 unreachable=0 failed=1
vmx2 : ok=1 changed=0 unreachable=0 failed=1
" succeeded: false

- Running the action without become_user, --> KO.

mab@mab-infra:~$ sudo st2 run ansible.playbook playbook=/home/mab/automation/ansible/junos_template/pb.bgp.2.yml cwd=/home/mab/automation/ansible . id: 5914aa657cae2209793f8de3 status: failed parameters: cwd: /home/mab/automation/ansible playbook: /home/mab/automation/ansible/junos_template/pb.bgp.2.yml result: failed: true return_code: 2 stderr: Executed command "/opt/stackstorm/virtualenvs/ansible/bin/ansible-playbook /home/mab/automation/ansible/junos_template/pb.bgp.2.yml" stdout: " PLAY [Create BGP junos configuration] **

TASK [Render BGP configuration for junos devices] ** ok: [vmx1] ok: [vmx2]

TASK [Push bgp configuration on devices] *** fatal: [vmx2]: FAILED! => {"changed": false, "failed": true, "msg": "junos-eznc >= 1.2.2 is required but does not appear to be installed. It can be installed using pip install junos-eznc"} fatal: [vmx1]: FAILED! => {"changed": false, "failed": true, "msg": "junos-eznc >= 1.2.2 is required but does not appear to be installed. It can be installed using pip install junos-eznc"} to retry, use: --limit @/home/mab/automation/ansible/junos_template/pb.bgp.2.retry

PLAY RECAP ***** vmx1 : ok=1 changed=0 unreachable=0 failed=1
vmx2 : ok=1 changed=0 unreachable=0 failed=1
" succeeded: false

- title 

- title 

[mab27]

output of the previous st2 commands (st2 run ansible.playbook ...) in verbose mode

st2_run_become-user_mab_vvvv.txt st2_run_become-user_stanley_vvvv.txt st2_run_become-user_none_vvvv.txt

[mab27]

• Running ansible-playbook command in the st2 ansible virtual env --> OK

  
  mab@mab-infra:~$ /opt/stackstorm/virtualenvs/ansible/bin/ansible-playbook /home/mab/automation/ansible/junos_template/pb.bgp.2.yml -i /home/mab/automation/ansible/hosts
  [DEPRECATION WARNING]: junos_template is kept for backwards compatibility but 
  usage is discouraged. The module documentation details page may explain more 
  about this rationale..
  This feature will be removed in a future release. 
  Deprecation warnings can be disabled by setting deprecation_warnings=False in 
  ansible.cfg.

PLAY [Create BGP junos configuration] **

TASK [Render BGP configuration for junos devices] ** ok: [vmx1] ok: [vmx2]

TASK [Push bgp configuration on devices] *** ok: [vmx2] ok: [vmx1]

PLAY [Wait for peers to establish connections] *****

TASK [pause] *** Pausing for 1 seconds (ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort) ok: [localhost]

PLAY [Check bgp states] ****

TASK [Check bgp peers states] ** ok: [vmx2] => (item={u'interface': u'ge-0/0/1', u'asn': 101, u'peer_ip': u'172.16.0.30'}) ok: [vmx1] => (item={u'interface': u'ge-0/0/1', u'asn': 102, u'peer_ip': u'172.16.0.40'})

PLAY RECAP ***** localhost : ok=1 changed=0 unreachable=0 failed=0
vmx1 : ok=3 changed=0 unreachable=0 failed=0
vmx2 : ok=3 changed=0 unreachable=0 failed=0

[mab27]

• Output of the previous ansible-playbook command in verbose mode

ansible-playbook__-vvv.txt --> OK

[mab27]

mab@mab-infra:~$ sudo st2 run ansible.playbook playbook=/home/mab/automation/ansible/junos_netconf/pb.yml cwd=/home/mab/automation/ansible/ version=true
.
id: 5915cb5f7cae220ad8ce413b
status: succeeded
parameters: 
  cwd: /home/mab/automation/ansible/
  playbook: /home/mab/automation/ansible/junos_netconf/pb.yml
  version: true
result: 
  failed: false
  return_code: 0
  stderr: ''
  stdout: "ansible-playbook 2.2.0.0
  config file = /home/mab/automation/ansible/ansible.cfg
  configured module search path = Default w/o overrides"
  succeeded: true

[arm4b]

As said before, help me to isolate & reproduce the problem with Ansible ad-hoc commands, instead of running entire playbook. Just logs are not so helpful, since I'm missing context & different environment.

For example, this works for me on a clean machine. We're trying to understand if custom Ansible module with pip package works or not:

# install pip dependency in pack virtualenv
sudo /opt/stackstorm/virtualenvs/ansible/bin/pip install junos-eznc

# install the role with custom module
st2 run ansible.galaxy.install roles=Juniper.junos

# run pure ansible virtualenv command
/opt/stackstorm/virtualenvs/ansible/bin/ansible all -i 'localhost,' -c local --module-name=junos_get_facts --module-path=/etc/ansible/roles/Juniper.junos/library -vvvv

# run st2 command equivalent
st2 run ansible.command connection=local inventory_file='localhost,' hosts=all module_name=junos_get_facts module_path=/etc/ansible/roles/Juniper.junos/library verbose=vvvv

Please run this ^^ to understand if we have any diff in results. Eg. in my env the junos_get_facts Ansible custom module worked meaning junos-eznc pip package is picked up.

[mab27]

In my case the one task (in the playbook) that is failing is this one:

- name: Push bgp configuration on devices
       junos_template:
       provider:
          host: "{{ credentials.host }}"
          username: "{{ credentials.username }}"
          ssh_keyfile: "{{ credentials.ssh_key }}"
          src: "{{playbook_dir}}/{{ inventory_hostname }}.conf"
        src: "{{playbook_dir}}/{{ inventory_hostname }}.conf"

[mab27]

Not sure how to translate this in a ansible.command command, as this particular task has some parameters that does not appear as available parameters for the ansible.command action (speacking st2 wise), I am thinking about provider, and src so I assume it would be something starting like this (but incomplete):

sudo st2 run ansible.command connection=local cwd=/home/mab/automation/ansible/ hosts=vmx module_name=junos_template verbose=vvvv

[mab27]

So I’m unable for now to reproduce precisely what you want … But more importantly, I don’t know how to find the module path. As this task relies on a core module of Ansible (junos_template), I don’t need ansible galaxy to install it. And going to the /etc/ansible directory, I don’t find it … which is strange isn’t it

mab@mab-infra:/etc/ansible/roles/Juniper.junos$ tree
.
├── callback_plugins
│   ├── jsnapy.py
│   └── jsnapy.pyc
├── COPYRIGHT
├── Dockerfile
├── docs
│   ├── ansible2rst.py
│   ├── conf.py
│   ├── docreq.txt
│   ├── juniper.png
│   └── rst.j2
├── env-setup
├── library
│   ├── __init__.py
│   ├── junos_cli
│   ├── junos_commit
│   ├── junos_get_config
│   ├── junos_get_facts
│   ├── junos_get_factsc
│   ├── junos_get_table
│   ├── junos_install_config
│   ├── junos_install_os
│   ├── junos_jsnapy
│   ├── junos_ping
│   ├── junos_rollback
│   ├── junos_rpc
│   ├── junos_shutdown
│   ├── junos_srx_cluster
│   └── junos_zeroize
├── LICENSE
├── meta
│   └── main.yml
├── README.md
├── requirements.txt
├── setup.py
├── tests
│   ├── ansible.cfg
│   ├── junos_jsnapy
│   │   ├── add_loopback.set
│   │   ├── delete_loopback.set
│   │   ├── test_junos_storage.yml
│   │   ├── test_loopback.yml
│   │   └── test_version.yml
│   ├── pb.junos_get_facts.yaml
│   ├── pb.junos_jsnapy.yaml
│   ├── pb.junos_ping.yaml
│   ├── pb.rav.token.app_stop.yaml
│   ├── pb.rav.token.create-deploy.yaml
│   ├── pb.rav.token.fqdn_get.yaml
│   └── ravello.ini
├── tools
│   └── sw_upgrade
└── version.py

7 directories, 46 files

[arm4b]

Using junos_template clarifies your env more, since it's core module and I don't need to install any custom Ansible role with plugins. So one step forward here in understanding the requirements.

According to https://docs.ansible.com/ansible/junos_template_module.html junos_template renamed to junos_config.

I'm on latest Ansible 2.3. Can you switch to latest Ansible and give me output of the following command (junos_config):

/opt/stackstorm/virtualenvs/ansible/bin/ansible localhost --inventory-file='localhost,' -vvvv --connection=local --module-name=junos_config

in your environment?

[mab27]

I was running version 2.2. I removed and re-installed the pack using st pack install, which lead to version 2.3 being installed

mab@mab-infra:/opt/stackstorm/packs$ /opt/stackstorm/virtualenvs/ansible/bin/ansible --version
ansible 2.3.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

And here's the result of the command you asked:

mab@mab-infra:/opt/stackstorm/packs$ /opt/stackstorm/virtualenvs/ansible/bin/ansible localhost --inventory-file='localhost,' -vvvv --connection=local --module-name=junos_config
Using /etc/ansible/ansible.cfg as config file
Set default localhost to localhost
Loading callback plugin minimal of type stdout, v2.0 from /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc
META: ran handlers
<localhost> using connection plugin netconf
<localhost> socket_path: /home/mab/.ansible/pc/23b443a485
localhost | FAILED! => {
    "changed": false, 
    "failed": true, 
    "msg": "unable to open shell. Please see: https://docs.ansible.com/ansible/network_debug_troubleshooting.html#unable-to-open-shell", 
    "rc": 1
}

[arm4b]

Try to debug further as suggested by Ansible:

• https://docs.ansible.com/ansible/network_debug_troubleshooting.html#unable-to-open-shell
• https://docs.ansible.com/ansible/network_debug_troubleshooting.html#enable-network-logging

to get more specific error message.