[Bug]: network_cable idempotency broken with dcim.consoleserverport and dcim.consoleport

allan-silverstein commented 1 year ago

Ansible NetBox Collection version

v3.13.0

Ansible version

ansible [core 2.15.0]
  config file = /home/asilver/pyprojects/aoe-lab-network-mgmt/ansible/netbox/ansible.cfg
  configured module search path = ['/home/asilver/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/asilver/pyvenv/ansible_netbox2/lib/python3.10/site-packages/ansible
  ansible collection location = /home/asilver/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/asilver/pyvenv/ansible_netbox2/bin/ansible
  python version = 3.10.1 (main, Jan 10 2022, 12:18:42) [GCC 9.3.0] (/home/asilver/pyvenv/ansible_netbox2/bin/python3.10)
  jinja version = 3.1.2
  libyaml = True

NetBox version

v3.5.2

Python version

3.10

Steps to Reproduce

Running this simple playbook the first time works fine and the cable connection between dcim.consoleserverport and dcim.consoleport is created. Running subsequent times results in errors. This seems to be related to https://github.com/netbox-community/ansible_modules/issues/989

- name: "Test for Populating Cables in Netbox Database"
  connection: local
  hosts: localhost
  gather_facts: false

  vars:
    install_state: present
    netbox_url: "https://fra04-inf09-k8slb.apexoe.fln.lab.dell.com:9443/"
    netbox_token: "{{ lookup('env', 'NETBOX_TOKEN') }}"

  tasks:
    - name: "Add single Console cable"
      netbox.netbox.netbox_cable:
        netbox_url: "{{ netbox_url }}"
        netbox_token: "{{ netbox_token }}"
        validate_certs: no
        data:
          termination_a:
            device: fra01-wls01-acs01
            name: Serial 1
          termination_a_type: dcim.consoleserverport
          termination_b:
            device: fra01-bls01-msw01
            name: Console 0
          termination_b_type: dcim.consoleport
        state: present

Expected Behavior

Idempotency should work where the same playbook can be run multiple times without adding the cable if the cable all ready exists.

Observed Behavior

Here is what happens when running it. Note, there are warning cert errors in module_stderr that show up which are not related to the issue. The error is at the end of module_stderr:

Exception: console-server-ports not found in API_APPS_ENDPOINTS\n",

(ansible_netbox2) asilver@dev-ubuntu-200403-gui-vm-3:~/pyprojects/aoe-lab-network-mgmt/ansible/netbox$ ansible-playbook netbox_load_cable_test.yaml -vvvv
ansible-playbook [core 2.15.0]
  config file = /home/asilver/pyprojects/aoe-lab-network-mgmt/ansible/netbox/ansible.cfg
  configured module search path = ['/home/asilver/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/asilver/pyvenv/ansible_netbox2/lib/python3.10/site-packages/ansible
  ansible collection location = /home/asilver/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/asilver/pyvenv/ansible_netbox2/bin/ansible-playbook
  python version = 3.10.1 (main, Jan 10 2022, 12:18:42) [GCC 9.3.0] (/home/asilver/pyvenv/ansible_netbox2/bin/python3.10)
  jinja version = 3.1.2
  libyaml = True
Using /home/asilver/pyprojects/aoe-lab-network-mgmt/ansible/netbox/ansible.cfg as config file
setting up inventory plugins
Loading collection ansible.builtin from 
host_list declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
script declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
yaml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
ini declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
toml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
Loading collection netbox.netbox from /home/asilver/.ansible/collections/ansible_collections/netbox/netbox
Loading callback plugin default of type stdout, v2.0 from /home/asilver/pyvenv/ansible_netbox2/lib/python3.10/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: netbox_load_cable_test.yaml ***************************************************************************************************************************************************************************************************************************************************
Positional arguments: netbox_load_cable_test.yaml
verbosity: 4
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/etc/ansible/hosts',)
forks: 5
1 plays in netbox_load_cable_test.yaml

PLAY [Test for Populating Cables in Netbox Database] ************************************************************************************************************************************************************************************************************************************

TASK [Add single Console cable] *********************************************************************************************************************************************************************************************************************************************************
task path: /home/asilver/pyprojects/aoe-lab-network-mgmt/ansible/netbox/netbox_load_cable_test.yaml:28
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: asilver
<127.0.0.1> EXEC /bin/sh -c 'echo ~asilver && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/asilver/.ansible/tmp `"&& mkdir "` echo /home/asilver/.ansible/tmp/ansible-tmp-1686083642.266194-2403956-232681313629586 `" && echo ansible-tmp-1686083642.266194-2403956-232681313629586="` echo /home/asilver/.ansible/tmp/ansible-tmp-1686083642.266194-2403956-232681313629586 `" ) && sleep 0'
Using module file /home/asilver/.ansible/collections/ansible_collections/netbox/netbox/plugins/modules/netbox_cable.py
<127.0.0.1> PUT /home/asilver/.ansible/tmp/ansible-local-2403949fdt8vbif/tmpu_n30uob TO /home/asilver/.ansible/tmp/ansible-tmp-1686083642.266194-2403956-232681313629586/AnsiballZ_netbox_cable.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/asilver/.ansible/tmp/ansible-tmp-1686083642.266194-2403956-232681313629586/ /home/asilver/.ansible/tmp/ansible-tmp-1686083642.266194-2403956-232681313629586/AnsiballZ_netbox_cable.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/home/asilver/pyvenv/ansible_netbox2/bin/python3.10 /home/asilver/.ansible/tmp/ansible-tmp-1686083642.266194-2403956-232681313629586/AnsiballZ_netbox_cable.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/asilver/.ansible/tmp/ansible-tmp-1686083642.266194-2403956-232681313629586/ > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "/home/asilver/pyvenv/ansible_netbox2/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'fra04-inf09-k8slb.apexoe.fln.lab.dell.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/asilver/pyvenv/ansible_netbox2/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'fra04-inf09-k8slb.apexoe.fln.lab.dell.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/asilver/pyvenv/ansible_netbox2/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'fra04-inf09-k8slb.apexoe.fln.lab.dell.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/asilver/pyvenv/ansible_netbox2/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'fra04-inf09-k8slb.apexoe.fln.lab.dell.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/asilver/pyvenv/ansible_netbox2/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'fra04-inf09-k8slb.apexoe.fln.lab.dell.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/asilver/pyvenv/ansible_netbox2/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'fra04-inf09-k8slb.apexoe.fln.lab.dell.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/asilver/pyvenv/ansible_netbox2/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'fra04-inf09-k8slb.apexoe.fln.lab.dell.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\nTraceback (most recent call last):\n  File \"/home/asilver/.ansible/tmp/ansible-tmp-1686083642.266194-2403956-232681313629586/AnsiballZ_netbox_cable.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/home/asilver/.ansible/tmp/ansible-tmp-1686083642.266194-2403956-232681313629586/AnsiballZ_netbox_cable.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/asilver/.ansible/tmp/ansible-tmp-1686083642.266194-2403956-232681313629586/AnsiballZ_netbox_cable.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.netbox.netbox.plugins.modules.netbox_cable', init_globals=dict(_module_fqn='ansible_collections.netbox.netbox.plugins.modules.netbox_cable', _modlib_path=modlib_path),\n  File \"/opt/python-3.10.1/lib/python3.10/runpy.py\", line 209, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/opt/python-3.10.1/lib/python3.10/runpy.py\", line 96, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/opt/python-3.10.1/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_j475nedm/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_cable.py\", line 364, in <module>\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_j475nedm/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_cable.py\", line 360, in main\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_j475nedm/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_dcim.py\", line 229, in run\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_j475nedm/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1419, in _ensure_object_exists\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_j475nedm/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1370, in _update_netbox_object\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_j475nedm/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1363, in _convert_termination\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_j475nedm/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1097, in _find_app\nException: console-server-ports not found in API_APPS_ENDPOINTS\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

PLAY RECAP ******************************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

(ansible_netbox2) asilver@dev-ubuntu-200403-gui-vm-3:~/pyprojects/aoe-lab-network-mgmt/ansible/netbox$

sc68cal commented 1 year ago

Verify that your ansible collection and netbox server versions match, since it is failing because the console-server-ports API endpoint is not being exposed by the NetBox server that you are connecting to.

allan-silverstein commented 1 year ago

Sorry about the delayed response.... Not sure about the correlation between ansible collection version and the Netbox server version but here is what I have: Netbox Server: 3.5.2 Ansible Galaxy netbox collection: 3.13.0

They obviously don't match but I believe that is the latest of each or pretty close to it. Are these the versions you were referring to or something else?

# /home/asilver/.ansible/collections/ansible_collections
Collection         Version
------------------ -------
ansible.netcommon  4.0.0  
ansible.posix      1.4.0  
ansible.utils      2.6.1  
community.general  5.8.0  
community.vmware   3.1.0  
dellemc.openmanage 6.3.0  
dellemc.os10       1.1.1  
dellemc.powerscale 1.9.0  
dellemc.powerstore 1.7.0  
netbox.netbox      3.13.0

vsurresh commented 4 months ago

I also have the exact same issue when using dcim.frontports.

dcim.interfaces however do work okay.

netbox-community / ansible_modules