Open mab27 opened 7 years ago
Following arma suggestion:
Try to install missing pip module in Ansible pack virtualenv:
/opt/stackstorm/virtualenvs/ansible/bin/pip install
pip install dependencies in the st2 ansible virtualenv:
mab@mab-infra:~/automation/ansible$ sudo /opt/stackstorm/virtualenvs/ansible/bin/pip install jxmlease
[sudo] password for mab:
The directory '/home/mab/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/mab/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting jxmlease
Downloading jxmlease-1.0.1-py2.py3-none-any.whl
Installing collected packages: jxmlease
Successfully installed jxmlease-1.0.1
mab@mab-infra:~/automation/ansible$ sudo /opt/stackstorm/virtualenvs/ansible/bin/pip install junos-eznc
The directory '/home/mab/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/mab/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting junos-eznc
Downloading junos-eznc-2.1.2.tar.gz (103kB)
100% |████████████████████████████████| 112kB 861kB/s
Collecting lxml>=3.2.4 (from junos-eznc)
Downloading lxml-3.7.3-cp27-cp27mu-manylinux1_x86_64.whl (6.8MB)
100% |████████████████████████████████| 6.8MB 99kB/s
Collecting ncclient>=0.5.3 (from junos-eznc)
Requirement already satisfied: paramiko>=1.15.2 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from junos-eznc)
Collecting scp>=0.7.0 (from junos-eznc)
Downloading scp-0.10.2-py2.py3-none-any.whl
Requirement already satisfied: jinja2>=2.7.1 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from junos-eznc)
Requirement already satisfied: PyYAML>=3.10 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from junos-eznc)
Collecting netaddr (from junos-eznc)
Downloading netaddr-0.7.19-py2.py3-none-any.whl (1.6MB)
100% |████████████████████████████████| 1.6MB 453kB/s
Requirement already satisfied: six in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from junos-eznc)
Collecting pyserial (from junos-eznc)
Downloading pyserial-3.3-py2.py3-none-any.whl (189kB)
100% |████████████████████████████████| 194kB 612kB/s
Requirement already satisfied: jxmlease in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from junos-eznc)
Requirement already satisfied: setuptools>0.6 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from ncclient>=0.5.3->junos-eznc)
Requirement already satisfied: pyasn1>=0.1.7 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from paramiko>=1.15.2->junos-eznc)
Requirement already satisfied: cryptography>=1.1 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from paramiko>=1.15.2->junos-eznc)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from jinja2>=2.7.1->junos-eznc)
Requirement already satisfied: appdirs>=1.4.0 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from setuptools>0.6->ncclient>=0.5.3->junos-eznc)
Requirement already satisfied: packaging>=16.8 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from setuptools>0.6->ncclient>=0.5.3->junos-eznc)
Requirement already satisfied: ipaddress in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from cryptography>=1.1->paramiko>=1.15.2->junos-eznc)
Requirement already satisfied: idna>=2.1 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from cryptography>=1.1->paramiko>=1.15.2->junos-eznc)
Requirement already satisfied: asn1crypto>=0.21.0 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from cryptography>=1.1->paramiko>=1.15.2->junos-eznc)
Requirement already satisfied: enum34 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from cryptography>=1.1->paramiko>=1.15.2->junos-eznc)
Requirement already satisfied: cffi>=1.4.1 in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from cryptography>=1.1->paramiko>=1.15.2->junos-eznc)
Requirement already satisfied: pyparsing in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from packaging>=16.8->setuptools>0.6->ncclient>=0.5.3->junos-eznc)
Requirement already satisfied: pycparser in /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages (from cffi>=1.4.1->cryptography>=1.1->paramiko>=1.15.2->junos-eznc)
Installing collected packages: lxml, ncclient, scp, netaddr, pyserial, junos-eznc
Running setup.py install for junos-eznc ... done
Successfully installed junos-eznc-2.1.2 lxml-3.7.3 ncclient-0.5.3 netaddr-0.7.19 pyserial-3.3 scp-0.10.2
pip list for the st2 ansible virtualenv (junos-eznc is now in):
mab@mab-infra:~/automation/ansible$ sudo /opt/stackstorm/virtualenvs/ansible/bin/pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
ansible (2.2.0.0)
appdirs (1.4.3)
asn1crypto (0.22.0)
cffi (1.10.0)
cryptography (1.8.1)
enum34 (1.1.6)
idna (2.5)
ipaddress (1.0.18)
Jinja2 (2.9.6)
junos-eznc (2.1.2)
jxmlease (1.0.1)
lxml (3.7.3)
MarkupSafe (1.0)
ncclient (0.5.3)
netaddr (0.7.19)
packaging (16.8)
paramiko (2.1.2)
pip (9.0.1)
pyasn1 (0.2.3)
pycparser (2.17)
pycrypto (2.6.1)
pyparsing (2.2.0)
pyserial (3.3)
PyYAML (3.12)
scp (0.10.2)
setuptools (35.0.2)
six (1.10.0)
wheel (0.29.0)
The directory '/home/mab/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
mab@mab-infra:~/automation/ansible$
Still the playbook fails with the exact same error message which is: junos-eznc >= 1.2.2 is required but does not appear to be installed.
Am I missing something with permissions ? users ? cache ?
sudo st2 run ansible.playbook playbook=/home/mab/automation/ansible/junos_template/pb.bgp.2.yml cwd=/home/mab/automation/ansible
.
id: 590f417d7cae220956aeb04e
status: failed
parameters:
cwd: /home/mab/automation/ansible
playbook: /home/mab/automation/ansible/junos_template/pb.bgp.2.yml
result:
failed: true
return_code: 2
stderr: Executed command "/opt/stackstorm/virtualenvs/ansible/bin/ansible-playbook /home/mab/automation/ansible/junos_template/pb.bgp.2.yml"
stdout: "
PLAY [Create BGP junos configuration] ******************************************
TASK [Render BGP configuration for junos devices] ** ok: [vmx1] ok: [vmx2]
TASK [Push bgp configuration on devices] ***
fatal: [vmx1]: FAILED! => {"changed": false, "failed": true, "msg": "junos-eznc >= 1.2.2 is required but does not appear to be installed. It can be installed using pip install junos-eznc
"}
fatal: [vmx2]: FAILED! => {"changed": false, "failed": true, "msg": "junos-eznc >= 1.2.2 is required but does not appear to be installed. It can be installed using pip install junos-eznc
"}
to retry, use: --limit @/home/mab/automation/ansible/junos_template/pb.bgp.2.retry
PLAY RECAP *****
vmx1 : ok=1 changed=0 unreachable=0 failed=1
vmx2 : ok=1 changed=0 unreachable=0 failed=1
"
succeeded: false
As said in Slack, the first step is to install missing pip
modules (jnpr.junos) in Ansible pack virtualenv and prepare environment. That's because StackStorm packs operate in their own isolated Python virtualenv.
Here is my start:
# install pip dependency
sudo /opt/stackstorm/virtualenvs/ansible/bin/pip install junos-eznc
# install custom Ansible module
st2 run ansible.galaxy.install roles=Juniper.junos
After that I'd suggest to debug your specific failed task with Ansible ad-hoc command manually. StackStorm ansible
pack is just an abstraction around the Ansible binaries like ansible
, ansible-playbook
, ansible-galaxy
.
For example this failed for me:
/opt/stackstorm/virtualenvs/ansible/bin/ansible all -i 'localhost,' -c local -vvvv --module-name=junos_get_facts
No config file found; using defaults
Set default localhost to localhost
Loading callback plugin minimal of type stdout, v2.0 from /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc
META: ran handlers
The full traceback is:
Traceback (most recent call last):
File "/opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 125, in run
res = self._execute()
File "/opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 521, in _execute
result = self._handler.run(task_vars=variables)
File "/opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages/ansible/plugins/action/junos.py", line 50, in run
module = module_loader._load_module_source(self._task.action, module_loader.find_plugin(self._task.action))
File "/opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages/ansible/plugins/__init__.py", line 337, in _load_module_source
with open(path, 'rb') as module_file:
TypeError: coercing to Unicode: need string or buffer, NoneType found
localhost | FAILED! => {
"failed": true,
"msg": "Unexpected failure during module execution.",
"stdout": ""
}
From the above error looks like it couldn't find path to the module.
My second try with --module-path=/etc/ansible/roles/Juniper.junos/library
:
/opt/stackstorm/virtualenvs/ansible/bin/ansible all -i 'localhost,' -c local -vvvv --module-name=junos_get_facts --module-path=/etc/ansible/roles/Juniper.junos/library
No config file found; using defaults
Set default localhost to localhost
Loading callback plugin minimal of type stdout, v2.0 from /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc
META: ran handlers
Using module file /etc/ansible/roles/Juniper.junos/library/junos_get_facts
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: ubuntu
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1494175104.31-241507766167253 `" && echo ansible-tmp-1494175104.31-241507766167253="` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1494175104.31-241507766167253 `" ) && sleep 0'
<localhost> PUT /tmp/tmpezir6u TO /home/ubuntu/.ansible/tmp/ansible-tmp-1494175104.31-241507766167253/junos_get_facts
<localhost> EXEC /bin/sh -c 'chmod u+x /home/ubuntu/.ansible/tmp/ansible-tmp-1494175104.31-241507766167253/ /home/ubuntu/.ansible/tmp/ansible-tmp-1494175104.31-241507766167253/junos_get_facts && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /home/ubuntu/.ansible/tmp/ansible-tmp-1494175104.31-241507766167253/junos_get_facts; rm -rf "/home/ubuntu/.ansible/tmp/ansible-tmp-1494175104.31-241507766167253/" > /dev/null 2>&1 && sleep 0'
localhost | FAILED! => {
"changed": false,
"failed": true,
"invocation": {
"module_args": {
"port": 830,
"user": "ubuntu"
}
},
"msg": "missing required arguments: host"
}
^^ that looks like success. In my example adding option in ansible.cfg
or using --module-path=
equivalent in Ansile pack so it could find the path to custom Ansible modules should work.
So like in my case, you probably need a bit deeper debugging. Hope that helps. At least try the logic above and provide logs what you see.
It sounds like we need per-pack virtualenv_opts. Does such a thing exist?
/etc/st2/st2.conf
has:
[actionrunner]
virtualenv_opts = --always-copy
and st2.conf.sample shows:
virtualenv_opts = --system-site-packages
That would allow ansible to access modules installed in the system, but might have unintended reprecussions for other packs. It would be excellent if the ansible pack could say "I need --enable-site-packages"
As a wrapper around the CLI, the ansible pack shouldn't really care which python or virtualenv is used to run ansible. What if, instead of installing ansible in the pack virtualenv, we used the system's ansible binary. A pre-requisite, then, would be to install ansible on the stackstorm system. Then, whoever is using a playbook that requires additional python packages, or other software, can just install them on the system.
Other possible ways to make the ansible pack behave better in the face of additional dependencies:
Add an action paramater to specify which ansible to use (system, or some path, other than the one in the pack):
why not make it an optional parameter on the action?
use_system_ansible
or maybeansible_path
then you get best of both worlds you get the "just works" functionality out of the box, then the option to customize/extend later -- @nmaludy in slack
Or add pack configuration to install additional python packages in the pack virtualenv:
you could also maybe list the additional python packages you need in the pack's config and have a "setup" action that initializes the ansible pack's virtualenv with the additional things defined in the config -- @nmaludy in slack
Install Ansible Roles from galaxy
Install python libraries required for the role:
Ansible configuration file:
Ansible version:
Ansible playbook details:
The playbook uses "connection: local". so the jnpr.junos python library has to be installed locally (hence the above pip and python commands)
Ansible variables:
Ansible playbook execution using ansible-playbook command (OK):
Ansible pack for st2 installed:
Verify the details st2 ansible pack is going to use:
Ansible playbook execution using st2 Ansible pack (KO):
TASK [create inventory directory] ** ok: [localhost]
PLAY [Get Facts] ***
TASK [remove host from inventory directory] **** ok: [vmx1] ok: [vmx2]
TASK [Retrieve information from devices running Junos] ***** fatal: [vmx2]: FAILED! => {"changed": false, "failed": true, "msg": "ImportError: No module named jnpr.junos"} fatal: [vmx1]: FAILED! => {"changed": false, "failed": true, "msg": "ImportError: No module named jnpr.junos"} to retry, use: --limit @/home/mab/automation/ansible/junos_get_facts/pb.retry
PLAY RECAP ***** localhost : ok=1 changed=0 unreachable=0 failed=0
vmx1 : ok=1 changed=0 unreachable=0 failed=1
vmx2 : ok=1 changed=0 unreachable=0 failed=1
" succeeded: false
mab@mab-infra:~/automation/ansible$ sudo st2 run ansible.playbook playbook=/home/mab/automation/ansible/junos_get_facts/pb.yml cwd=/home/mab/automation/ansible/ verbose=vvvv . id: 590f2fe17cae220956aeb033 status: failed parameters: cwd: /home/mab/automation/ansible/ playbook: /home/mab/automation/ansible/junos_get_facts/pb.yml verbose: vvvv result: failed: true return_code: 2 stderr: Executed command "/opt/stackstorm/virtualenvs/ansible/bin/ansible-playbook -vvvv /home/mab/automation/ansible/junos_get_facts/pb.yml" stdout: "Using /home/mab/automation/ansible/ansible.cfg as config file Loading callback plugin default of type stdout, v2.0 from /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages/ansible/plugins/callback/init.pyc Loading callback plugin jsnapy of type aggregate, v2.0 from /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages/ansible/plugins/callback/init.pyc
PLAYBOOK: pb.yml *** 2 plays in /home/mab/automation/ansible/junos_get_facts/pb.yml
PLAY [create inventory directory] ** META: ran handlers
TASK [create inventory directory] ** task path: /home/mab/automation/ansible/junos_get_facts/pb.yml:8 Using module file /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages/ansible/modules/files/file.py <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root <127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0' <127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "
echo /root/.ansible/tmp/ansible-tmp-1494167521.98-137850814807689
" && echo ansible-tmp-1494167521.98-137850814807689="echo /root/.ansible/tmp/ansible-tmp-1494167521.98-137850814807689
" ) && sleep 0' <127.0.0.1> PUT /tmp/tmpDz9pYO TO /root/.ansible/tmp/ansible-tmp-1494167521.98-137850814807689/file.py <127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1494167521.98-137850814807689/ /root/.ansible/tmp/ansible-tmp-1494167521.98-137850814807689/file.py && sleep 0' <127.0.0.1> EXEC /bin/sh -c '/opt/stackstorm/virtualenvs/ansible/bin/python /root/.ansible/tmp/ansible-tmp-1494167521.98-137850814807689/file.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1494167521.98-137850814807689/" > /dev/null 2>&1 && sleep 0' ok: [localhost] => { "changed": false, "diff": { "after": { "path": "/home/mab/automation/ansible/junos_get_facts/inventory" }, "before": { "path": "/home/mab/automation/ansible/junos_get_facts/inventory" } }, "gid": 0, "group": "root", "invocation": { "module_args": { "attributes": null, "backup": null, "content": null, "delimiter": null, "diff_peek": null, "directory_mode": null, "follow": false, "force": false, "group": null, "mode": null, "original_basename": null, "owner": null, "path": "/home/mab/automation/ansible/junos_get_facts/inventory", "recurse": false, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": null, "state": "directory", "unsafe_writes": null, "validate": null } }, "mode": "0755", "owner": "root", "path": "/home/mab/automation/ansible/junos_get_facts/inventory", "size": 4096, "state": "directory", "uid": 0 } META: ran handlers META: ran handlersPLAY [Get Facts] *** META: ran handlers
TASK [remove host from inventory directory] **** task path: /home/mab/automation/ansible/junos_get_facts/pb.yml:20 Using module file /opt/stackstorm/virtualenvs/ansible/lib/python2.7/site-packages/ansible/modules/files/file.py