search

ansible-collections / community.grafana

Grafana Collection for Ansible
http://galaxy.ansible.com/community/grafana
GNU General Public License v3.0
127 stars 82 forks source link

community.grafana.grafana_dashboard module fails with "file not found" #302

Closed bucovaina closed 1 year ago

bucovaina commented 1 year ago
SUMMARY

I'm trying to upload a dashboard with the file acr10r.json but I can't make the module install the dashboard. For some reason the module can't find the file. I have tried various things to make it work to no avail. The file originally resided in myansiblerepo/roles/rolename/files/acr10r.json (same rolename as the tasks/main.yml file that contains the task). First the path was set to "acr10r.json" like you would do with the Ansible builtin copy module. That didn't work, I had to specify the path. Then I tried, "files/acr10r.json", "../files/acr10r.json" and the entire path /home/user/Documents/.../.../rolename/files/acr10r.json), Then I tried moving the file to /tmp/ and chmod it to 777. But nothing seems to be working. I tried with the path surrounded by "", ' ' and without also to no avail.

ISSUE TYPE
COMPONENT NAME

grafana_dashboard

ANSIBLE VERSION
$ ansible --version
ansible [core 2.14.2]
  config file = /home/user/Documents/energy-monitoring-pve-ansible/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/user/.local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/user/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True
COLLECTION VERSION
$ ansible-galaxy collection list
[DEPRECATION WARNING]: DEFAULT_GATHER_TIMEOUT option, the module_defaults keyword is a more generic version and can apply to 
all calls to the M(ansible.builtin.gather_facts) or M(ansible.builtin.setup) actions, use module_defaults instead. This 
feature will be removed from ansible-core in version 2.18. Deprecation warnings can be disabled by setting 
deprecation_warnings=False in ansible.cfg.

# /usr/lib/python3/dist-packages/ansible_collections
Collection                Version
------------------------- -------
amazon.aws                1.4.0  
ansible.netcommon         1.5.0  
ansible.posix             1.1.1  
ansible.windows           1.4.0  
arista.eos                1.3.0  
awx.awx                   14.1.0 
azure.azcollection        1.4.0  
check_point.mgmt          1.0.6  
chocolatey.chocolatey     1.0.2  
cisco.aci                 1.1.1  
cisco.asa                 1.0.4  
cisco.intersight          1.0.10 
cisco.ios                 1.3.0  
cisco.iosxr               1.2.1  
cisco.meraki              2.2.0  
cisco.mso                 1.1.0  
cisco.nso                 1.0.3  
cisco.nxos                1.4.0  
cisco.ucs                 1.6.0  
cloudscale_ch.cloud       1.3.1  
community.aws             1.3.0  
community.azure           1.0.0  
community.crypto          1.4.0  
community.digitalocean    1.0.0  
community.docker          1.2.2  
community.fortios         1.0.0  
community.general         1.3.6  
community.google          1.0.0  
community.grafana         1.1.0  
community.hashi_vault     1.1.0  
community.hrobot          1.1.0  
community.kubernetes      1.1.1  
community.kubevirt        1.0.0  
community.libvirt         1.0.0  
community.mongodb         1.2.0  
community.mysql           1.2.0  
community.network         1.3.2  
community.okd             1.0.0  
community.postgresql      1.1.1  
community.proxysql        1.0.0  
community.rabbitmq        1.0.1  
community.routeros        1.1.0  
community.skydive         1.0.0  
community.vmware          1.7.0  
community.windows         1.3.0  
community.zabbix          1.2.0  
containers.podman         1.4.1  
cyberark.conjur           1.1.0  
cyberark.pas              1.0.5  
dellemc.os10              1.0.2  
dellemc.os6               1.0.6  
dellemc.os9               1.0.3  
f5networks.f5_modules     1.7.1  
fortinet.fortimanager     1.0.5  
fortinet.fortios          1.1.8  
frr.frr                   1.0.3  
gluster.gluster           1.0.1  
google.cloud              1.0.2  
hetzner.hcloud            1.2.1  
ibm.qradar                1.0.3  
infinidat.infinibox       1.2.4  
junipernetworks.junos     1.3.0  
mellanox.onyx             1.0.0  
netapp.aws                20.9.0 
netapp.elementsw          20.11.0
netapp.ontap              20.12.0
netapp_eseries.santricity 1.1.0  
netbox.netbox             1.2.1  
ngine_io.cloudstack       1.2.0  
ngine_io.exoscale         1.0.0  
ngine_io.vultr            1.1.0  
openstack.cloud           1.2.1  
openvswitch.openvswitch   1.1.0  
ovirt.ovirt               1.3.0  
purestorage.flasharray    1.6.2  
purestorage.flashblade    1.4.0  
servicenow.servicenow     1.0.4  
splunk.es                 1.0.2  
theforeman.foreman        1.5.1  
vyos.vyos                 1.1.1  
wti.remote                1.0.1  

# /home/user/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
community.general 6.4.0
CONFIGURATION
CONFIG_FILE() = /home/user/Documents/energy-monitoring-pve-ansible/ansible.cfg
DEFAULT_GATHER_TIMEOUT(/home/user/Documents/energy-monitoring-pve-ansible/ansible.cfg) = 30
DEFAULT_STDOUT_CALLBACK(/home/user/Documents/energy-monitoring-pve-ansible/ansible.cfg) = yaml
HOST_KEY_CHECKING(/home/user/Documents/energy-monitoring-pve-ansible/ansible.cfg) = False
OS / ENVIRONMENT

Ansible controller node: Debian 11.6.1 latest at time of writing Ansible controlled node: Idem but it's an LXC container in Proxmox, not a VM/bare metal machine.

STEPS TO REPRODUCE

So I run my ansible playbook as followd. The API key is freshly made with admin rights. I have tried various things with path: '/tmp/acr10r.json' . With/without apostrophe or double quotes. Full path relative path to main.yml, ... In this example, the file was effectively in /tmp/acr10r.json with 777 as permission bits to make absolutely sure it's not a permission problem.

ansible-playbook site.yml -i hosts --start-at-task "brokentask" # variables are defined and resolved correctly.
# cat brokentask (as an example)
# - name: brokentask
#  community.grafana.grafana_dashboard:
#    grafana_url: http://{{ grafanaip }}:3000
#    grafana_api_key: "{{ grafanaAPIKey }}"
#    state: present
#    commit_message: Updated by ansible
#    overwrite: yes
#    path: '/tmp/acr10r.json'
EXPECTED RESULTS

The dashboard to be installed like when I upload the exact same .json file to the web interface of the exact same instance and I can access it. Or at least an authentication failure that the API key is incorrect. My best guess is that the failure happens before the API key is tried.

ACTUAL RESULTS
task path: /home/user/Documents/energy-monitoring-pve-ansible/roles/grafana/tasks/main.yml:166
<192.168.0.3> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.3> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'ControlPath="/home/user/.ansible/cp/4fa6b25422"' 192.168.0.3 '/bin/sh -c '"'"'echo ~root && sleep 0'"'"''
<192.168.0.3> (0, b'/root\n', b'')
<192.168.0.3> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.3> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'ControlPath="/home/user/.ansible/cp/4fa6b25422"' 192.168.0.3 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1678255025.230129-112320-1228533377129 `" && echo ansible-tmp-1678255025.230129-112320-1228533377129="` echo /root/.ansible/tmp/ansible-tmp-1678255025.230129-112320-1228533377129 `" ) && sleep 0'"'"''
<192.168.0.3> (0, b'ansible-tmp-1678255025.230129-112320-1228533377129=/root/.ansible/tmp/ansible-tmp-1678255025.230129-112320-1228533377129\n', b'')
Using module file /usr/lib/python3/dist-packages/ansible_collections/community/grafana/plugins/modules/grafana_dashboard.py
<192.168.0.3> PUT /home/user/.ansible/tmp/ansible-local-112301ms0uolsf/tmp8y5nv9d8 TO /root/.ansible/tmp/ansible-tmp-1678255025.230129-112320-1228533377129/AnsiballZ_grafana_dashboard.py
<192.168.0.3> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'ControlPath="/home/user/.ansible/cp/4fa6b25422"' '[192.168.0.3]'
<192.168.0.3> (0, b'sftp> put /home/user/.ansible/tmp/ansible-local-112301ms0uolsf/tmp8y5nv9d8 /root/.ansible/tmp/ansible-tmp-1678255025.230129-112320-1228533377129/AnsiballZ_grafana_dashboard.py\n', b'')
<192.168.0.3> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.3> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'ControlPath="/home/user/.ansible/cp/4fa6b25422"' 192.168.0.3 '/bin/sh -c '"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-1678255025.230129-112320-1228533377129/ /root/.ansible/tmp/ansible-tmp-1678255025.230129-112320-1228533377129/AnsiballZ_grafana_dashboard.py && sleep 0'"'"''
<192.168.0.3> (0, b'', b'')
<192.168.0.3> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.3> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'ControlPath="/home/user/.ansible/cp/4fa6b25422"' -tt 192.168.0.3 '/bin/sh -c '"'"'/usr/bin/python3 /root/.ansible/tmp/ansible-tmp-1678255025.230129-112320-1228533377129/AnsiballZ_grafana_dashboard.py && sleep 0'"'"''
<192.168.0.3> (1, b'\r\n{"failed": true, "msg": "error : Can\'t load json file [Errno 2] No such file or directory: \'/tmp/acr10r.json\'", "exception": "  File \\"/tmp/ansible_community.grafana.grafana_dashboard_payload_55r9pr3u/ansible_community.grafana.grafana_dashboard_payload.zip/ansible_collections/community/grafana/plugins/modules/grafana_dashboard.py\\", line 525, in main\\n  File \\"/tmp/ansible_community.grafana.grafana_dashboard_payload_55r9pr3u/ansible_community.grafana.grafana_dashboard_payload.zip/ansible_collections/community/grafana/plugins/modules/grafana_dashboard.py\\", line 311, in grafana_create_dashboard\\n", "invocation": {"module_args": {"grafana_url": "http://192.168.0.3:3000", "grafana_api_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "state": "present", "commit_message": "Updated by ansible", "overwrite": true, "path": "/tmp/acr10r.json", "url": "http://192.168.0.3:3000", "use_proxy": true, "validate_certs": true, "url_username": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "url_password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "use_gssapi": false, "org_id": 1, "folder": "General", "dashboard_revision": "1", "client_cert": null, "client_key": null, "uid": null, "slug": null, "dashboard_id": null}}}\r\n', b'Shared connection to 192.168.0.3 closed.\r\n')
<192.168.0.3> Failed to connect to the host via ssh: Shared connection to 192.168.0.3 closed.
<192.168.0.3> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.3> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'ControlPath="/home/user/.ansible/cp/4fa6b25422"' 192.168.0.3 '/bin/sh -c '"'"'rm -f -r /root/.ansible/tmp/ansible-tmp-1678255025.230129-112320-1228533377129/ > /dev/null 2>&1 && sleep 0'"'"''
<192.168.0.3> (0, b'', b'')
The full traceback is:
  File "/tmp/ansible_community.grafana.grafana_dashboard_payload_55r9pr3u/ansible_community.grafana.grafana_dashboard_payload.zip/ansible_collections/community/grafana/plugins/modules/grafana_dashboard.py", line 525, in main
  File "/tmp/ansible_community.grafana.grafana_dashboard_payload_55r9pr3u/ansible_community.grafana.grafana_dashboard_payload.zip/ansible_collections/community/grafana/plugins/modules/grafana_dashboard.py", line 311, in grafana_create_dashboard
fatal: [grafana]: FAILED! => changed=false 
  invocation:
    module_args:
      client_cert: null
      client_key: null
      commit_message: Updated by ansible
      dashboard_id: null
      dashboard_revision: '1'
      folder: General
      grafana_api_key: VALUE_SPECIFIED_IN_NO_LOG_PARAMETER
      grafana_url: http://192.168.0.3:3000
      org_id: 1
      overwrite: true
      path: /tmp/acr10r.json
      slug: null
      state: present
      uid: null
      url: http://192.168.0.3:3000
      url_password: VALUE_SPECIFIED_IN_NO_LOG_PARAMETER
      url_username: VALUE_SPECIFIED_IN_NO_LOG_PARAMETER
      use_gssapi: false
      use_proxy: true
      validate_certs: true
  msg: 'error : Can''t load json file [Errno 2] No such file or directory: ''/tmp/acr10r.json'''
bucovaina commented 1 year ago

facepalm ...

The file was on the controller node. I tried scp-ing the acr10r.json file to the controlled node and low and behold, it works ...

bucovaina commented 1 year ago

Please go ahead and close as PEBKAC

rrey commented 1 year ago

I was wondering how the corresponding code could actually fail :D

rrey commented 1 year ago

community.grafana 1.1.0 Latest on ansible galaxy is 1.5.4, you may want to bump your version.

bucovaina commented 1 year ago

Thanks for the suggestion, I might have a look at that :)

Yes, I don't know, maybe it's my lack of experience but it was not clear to me that the file had to be on the controlled node, not on the controller node. Is it worth mentioning that in the documentation or should any reasonable person know you have to scp/push the file first to the controlled node, then install it with this module? :D.

At least, I won't forget now, have been breaking my head over this for a few hours :)