netbox-community / ansible_modules

NetBox modules for Ansible using Ansible Collections
GNU General Public License v3.0
308 stars 198 forks source link

[Bug]: netbox.netbox.netbox_cable module causes exceptions on secondary playbook runs #1040

Open TGH-TH opened 11 months ago

TGH-TH commented 11 months ago

Ansible NetBox Collection version

v3.13.0

Ansible version

ansible [core 2.13.5]
  config file = /home/redacted/projects/ansible/NetBox/ansible.cfg
  configured module search path = ['/home/redacted/projects/ansible/NetBox/library']
  ansible python module location = /home/redacted/.local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/redacted/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.13 (main, Nov 16 2022, 15:31:39) [GCC 8.5.0 20210514 (Red Hat 8.5.0-15)]
  jinja version = 3.1.2
  libyaml = True

NetBox version

v3.5.6

Python version

3.9

Steps to Reproduce

Create a cable connecting a dcim.powerport and dcim.poweroutlet endpoints using the netbox.netbox.netbox_cable module. This runs fine and creates the cable however subsequent runs create an exception The error was: Exception: power-ports not found in API_APPS_ENDPOINTS

Additionally a similar albeit slightly different error occurs when doing the same above with dcim.powerfeed and dcim.powerport end points but the error this time is AttributeError: 'dict' object has no attribute 'endpoint'

I can work around this by creating a task before the offending task that deletes the cable first - this allows the second task to create the cable successfully

Expected Behavior

First run should produce a 'changed' task as the cable is created

second (and subsequent) run should produce a green 'not changed' task

Observed Behavior

First run should produce a 'changed' task as the cable is created

second (and subsequent) runs produce an exception Exception: power-ports not found in API_APPS_ENDPOINTS for dcim.powerport and dcim.poweroutlet combos and Exception: AttributeError: 'dict' object has no attribute 'endpoint' for dcim.powerport and dcim.powerfeed combos

playbook fails at this point.

slazer2au commented 8 months ago

I am hitting this in the latest release of the Ansible Collection with dcim.frontport but not with dcim.rearport Ansible NetBox Collection version v3.14.0 NetBox version v3.6.3

Front port.

failed: [ams1-g1-r1] (item={'label': 'AMS1-G1-R1-PP - AMS1-G1-R1-PP', 'termination_a_type': 'dcim.frontport', 'termination_a': {'device': 'AMS1-G1-R1-PP', 'name': '01-2'}, 'termination_b_type': 'dcim.frontport', 'termination_b': {'device': 'AMS1-G1-R1-PP', 'name': '03-3'}, 'type': 'smf', 'status': 'connected', 'length': 15, 'state': 'present'}) => {
    "ansible_loop_var": "item",
    "changed": false,
    "item": {
        "label": "AMS1-G1-R1-PP - AMS1-G1-R1-PP",
        "length": 15,
        "state": "present",
        "status": "connected",
        "termination_a": {
            "device": "AMS1-G1-R1-PP",
            "name": "01-2"
        },
        "termination_a_type": "dcim.frontport",
        "termination_b": {
            "device": "AMS1-G1-R1-PP",
            "name": "03-3"
        },
        "termination_b_type": "dcim.frontport",
        "type": "smf"
    },
    "module_stderr": "/home/panda/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.2.14'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/panda/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.2.14'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/panda/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.2.14'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/panda/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.2.14'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/panda/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.2.14'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/panda/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.2.14'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/panda/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.2.14'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/panda/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.2.14'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/panda/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.2.14'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\n/home/panda/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.2.14'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n  warnings.warn(\nTraceback (most recent call last):\n  File \"/home/panda/.ansible/tmp/ansible-tmp-1698049122.8236706-135-135049627389029/AnsiballZ_netbox_cable.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/home/panda/.ansible/tmp/ansible-tmp-1698049122.8236706-135-135049627389029/AnsiballZ_netbox_cable.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/panda/.ansible/tmp/ansible-tmp-1698049122.8236706-135-135049627389029/AnsiballZ_netbox_cable.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.netbox.netbox.plugins.modules.netbox_cable', init_globals=dict(_module_fqn='ansible_collections.netbox.netbox.plugins.modules.netbox_cable', _modlib_path=modlib_path),\n  File \"/usr/lib/python3.10/runpy.py\", line 224, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib/python3.10/runpy.py\", line 96, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_5tf3ang8/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_cable.py\", line 371, in <module>\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_5tf3ang8/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_cable.py\", line 367, in main\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_5tf3ang8/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_dcim.py\", line 229, in run\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_5tf3ang8/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1437, in _ensure_object_exists\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_5tf3ang8/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1388, in _update_netbox_object\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_5tf3ang8/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1381, in _convert_termination\n  File \"/tmp/ansible_netbox.netbox.netbox_cable_payload_5tf3ang8/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1115, in _find_app\nException: front-ports not found in API_APPS_ENDPOINTS\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

Rear port

ok: [ams1-g1-r0] => (item={'label': 'AMS1-G1-R0-PP01-MOD1 - AMS1-G1-R1-PP-MOD1', 'termination_a_type': 'dcim.rearport', 'termination_a': {'device': 'AMS1-G1-R0-PP01', 'name': 1}, 'termination_b_type': 'dcim.rearport', 'termination_b': {'device': 'AMS1-G1-R1-PP', 'name': '1'}, 'type': 'smf', 'status': 'connected', 'length': 15, 'state': 'present'}) => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "ansible_loop_var": "item",
    "cable": {
        "a_terminations": [
            {
                "object_id": 6,
                "object_type": "dcim.rearport"
            }
        ],
        "b_terminations": [
            {
                "object_id": 2,
                "object_type": "dcim.rearport"
            }
        ],
        "color": "",
        "comments": "",
        "created": "2023-10-17T09:56:59.994047Z",
        "custom_fields": {},
        "description": "",
        "display": "AMS1-G1-R0-PP01-MOD1 - AMS1-G1-R1-PP-MOD1",
        "id": 11,
        "label": "AMS1-G1-R0-PP01-MOD1 - AMS1-G1-R1-PP-MOD1",
        "last_updated": "2023-10-17T12:40:16.220558Z",
        "length": 15.0,
        "length_unit": "m",
        "status": "connected",
        "tags": [],
        "tenant": null,
        "type": "smf",
        "url": "https://192.168.2.14/api/dcim/cables/11/"
    },
    "changed": false,
    "invocation": {
        "module_args": {
            "cert": null,
            "data": {
                "color": null,
                "comments": null,
                "custom_fields": null,
                "description": null,
                "label": "AMS1-G1-R0-PP01-MOD1 - AMS1-G1-R1-PP-MOD1",
                "length": 15.0,
                "length_unit": "m",
                "status": "connected",
                "tags": null,
                "tenant": null,
                "termination_a": {
                    "device": "AMS1-G1-R0-PP01",
                    "name": 1
                },
                "termination_a_type": "dcim.rearport",
                "termination_b": {
                    "device": "AMS1-G1-R1-PP",
                    "name": "1"
                },
                "termination_b_type": "dcim.rearport",
                "type": "smf"
            },
            "netbox_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "netbox_url": "https://192.168.2.14",
            "query_params": null,
            "state": "present",
            "validate_certs": false
        }
    },
    "item": {
        "label": "AMS1-G1-R0-PP01-MOD1 - AMS1-G1-R1-PP-MOD1",
        "length": 15,
        "state": "present",
        "status": "connected",
        "termination_a": {
            "device": "AMS1-G1-R0-PP01",
            "name": 1
        },
        "termination_a_type": "dcim.rearport",
        "termination_b": {
            "device": "AMS1-G1-R1-PP",
            "name": "1"
        },
        "termination_b_type": "dcim.rearport",
        "type": "smf"
    },
    "msg": "cable dcim.rearport 1 <> dcim.rearport 1 already exists"
}
vsurresh commented 4 months ago

I have the same issue when using different dcim.type for term_a and term_b. For example, if both sides are the same type (dcim.interface) then I can run the playbook multiple types. However, if I use dcim.interface on one side and dcim.fronport on another side, the first run works okay but the subsequent runs fails.

NetBox - 3.7.1 Python - 3.9 Ansible - 2.11.3

Here is the error

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Exception: front-ports not found in API_APPS_ENDPOINTS
failed: [localhost] (item={'name': 'CORE-01 TO FW-02', 'cable_type': 'mmf-om3', 'color': '00ffff', 'length': 3, 'term_a': {'device': 'DC-Core-01', 'interface': 'Eth1/11', 'type': 'dcim.interface'}, 'term_b': {'device': 'patch_panel_dc_a1', 'interface': 'dc_a1_front_04', 'type': 'dcim.frontport'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"cable_type": "mmf-om3", "color": "00ffff", "length": 3, "name": "CORE-01 TO FW-02", "term_a": {"device": "DC-Core-01", "interface": "Eth1/11", "type": "dcim.interface"}, "term_b": {"device": "patch_panel_dc_a1", "interface": "dc_a1_front_04", "type": "dcim.frontport"}}, "module_stderr": "Traceback (most recent call last):\n  File \"/Users/sureshv/.ansible/tmp/ansible-tmp-1709299292.660817-71491-227641288328701/AnsiballZ_netbox_cable.py\", line 100, in <module>\n    _ansiballz_main()\n  File \"/Users/sureshv/.ansible/tmp/ansible-tmp-1709299292.660817-71491-227641288328701/AnsiballZ_netbox_cable.py\", line 92, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/Users/sureshv/.ansible/tmp/ansible-tmp-1709299292.660817-71491-227641288328701/AnsiballZ_netbox_cable.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.netbox.netbox.plugins.modules.netbox_cable', init_globals=dict(_module_fqn='ansible_collections.netbox.netbox.plugins.modules.netbox_cable', _modlib_path=modlib_path),\n  File \"/opt/homebrew/Cellar/python@3.9/3.9.18/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py\", line 225, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/opt/homebrew/Cellar/python@3.9/3.9.18/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/opt/homebrew/Cellar/python@3.9/3.9.18/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/var/folders/g9/7rxk0kqn4yg5cyv37d6_rhj9x4z3p8/T/ansible_netbox.netbox.netbox_cable_payload_un3piqeh/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_cable.py\", line 371, in <module>\n  File \"/var/folders/g9/7rxk0kqn4yg5cyv37d6_rhj9x4z3p8/T/ansible_netbox.netbox.netbox_cable_payload_un3piqeh/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_cable.py\", line 367, in main\n  File \"/var/folders/g9/7rxk0kqn4yg5cyv37d6_rhj9x4z3p8/T/ansible_netbox.netbox.netbox_cable_payload_un3piqeh/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_dcim.py\", line 225, in run\n  File \"/var/folders/g9/7rxk0kqn4yg5cyv37d6_rhj9x4z3p8/T/ansible_netbox.netbox.netbox_cable_payload_un3piqeh/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1481, in _ensure_object_exists\n  File \"/var/folders/g9/7rxk0kqn4yg5cyv37d6_rhj9x4z3p8/T/ansible_netbox.netbox.netbox_cable_payload_un3piqeh/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1435, in _update_netbox_object\n  File \"/var/folders/g9/7rxk0kqn4yg5cyv37d6_rhj9x4z3p8/T/ansible_netbox.netbox.netbox_cable_payload_un3piqeh/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1425, in _convert_termination\n  File \"/var/folders/g9/7rxk0kqn4yg5cyv37d6_rhj9x4z3p8/T/ansible_netbox.netbox.netbox_cable_payload_un3piqeh/ansible_netbox.netbox.netbox_cable_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 1157, in _find_app\nException: front-ports not found in API_APPS_ENDPOINTS\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Here is the playbook and vars.

tasks:
  - name: Create cables
    netbox.netbox.netbox_cable:
      netbox_url: http://10.10.10.10:8000
      netbox_token: TOKEN
      data:
        description: "{{ item.name }}"
        type: "{{ item.cable_type }}"
        color: "{{ item.color }}"
        length: "{{ item.length }}"
        length_unit: m
        termination_a_type: "{{ item.term_a.type }}"
        termination_a:
          device: "{{ item.term_a.device }}"
          name: "{{ item.term_a.interface }}"
        termination_b_type: "{{ item.term_b.type }}"
        termination_b:
          device: "{{ item.term_b.device }}"
          name: "{{ item.term_b.interface }}"
      state: present
    loop: "{{ connections }}"

---
connections:
  - name: DC-A1 to DC-B1
    cable_type: mmf-om3
    color: 00ffff
    length: 10
    term_a:
      device: patch_panel_dc_a1
      interface: dc_a1_rear_01
      type: dcim.rearport
    term_b:
      device: patch_panel_dc_b1
      interface: dc_b1_rear_01
      type: dcim.rearport
  - name: CORE-01 TO FW-01
    cable_type: mmf-om3
    color: 00ffff
    length: 3
    term_a:
      device: DC-Core-01
      interface: Eth1/10
      type: dcim.interface
    term_b:
      device: DC-FW-01
      interface: Ethernet1/1
      type: dcim.interface
  - name: CORE-02 TO FW-02
    cable_type: mmf-om3
    color: 00ffff
    length: 3
    term_a:
      device: DC-Core-02
      interface: Eth1/11
      type: dcim.interface
    term_b:
      device: DC-FW-02
      interface: Ethernet1/2
      type: dcim.interface
  - name: CORE-01 TO FW-02
    cable_type: mmf-om3
    color: 00ffff
    length: 3
    term_a:
      device: DC-Core-01
      interface: Eth1/11
      type: dcim.interface
    term_b:
      device: patch_panel_dc_a1
      interface: dc_a1_front_04
      type: dcim.frontport
  - name: CORE-01 TO FW-02
    cable_type: mmf-om3
    color: 00ffff
    length: 3
    term_a:
      device: DC-FW-02
      interface: Ethernet1/1
      type: dcim.interface
    term_b:
      device: patch_panel_dc_b1
      interface: dc_b1_front_04
      type: dcim.frontport
ellisgeek commented 2 months ago

This is effecting me on netbox 3.7.4 with netbox.netbox 1.17.0.

I believe this is caused by missing values in the API_APPS_ENDPOINTS and ENDPOINT_NAME_MAPPING dictionaries in plugins/module_utils/netbox_utils.py

In my case I was running into this error on the second run when connecting interfaces to front ports.

Adding an entry to both dicts for front-ports seems to have fixed my issue and I am able to complete a playbook run.

diff --git a/plugins/module_utils/netbox_utils.py b/plugins/module_utils/netbox_utils.py
index 8e9215e..6eda560 100644
--- a/plugins/module_utils/netbox_utils.py
+++ b/plugins/module_utils/netbox_utils.py
@@ -44,6 +44,7 @@ API_APPS_ENDPOINTS = dict(
         "console_ports": {},
         "console_port_templates": {},
         "console_server_ports": {},
+        "console-server-ports": {},
         "console_server_port_templates": {},
         "device_bays": {},
         "device_bay_templates": {},
@@ -51,6 +52,7 @@ API_APPS_ENDPOINTS = dict(
         "device_roles": {},
         "device_types": {},
         "front_ports": {},
+        "front-ports": {},
         "front_port_templates": {},
         "interfaces": {},
         "interface_templates": {},
@@ -67,6 +69,7 @@ API_APPS_ENDPOINTS = dict(
         "power_outlet_templates": {},
         "power_panels": {},
         "power_ports": {},
+        "power-ports": {},
         "power_port_templates": {},
         "racks": {},
         "rack_groups": {},
@@ -342,6 +345,7 @@ ENDPOINT_NAME_MAPPING = {
     "console_ports": "console_port",
     "console_port_templates": "console_port_template",
     "console_server_ports": "console_server_port",
+    "console-server-ports": "console_server_port",
     "console_server_port_templates": "console_server_port_template",
     "contacts": "contact",
     "contact_groups": "contact_group",
@@ -358,6 +362,7 @@ ENDPOINT_NAME_MAPPING = {
     "fhrp_groups": "fhrp_group",
     "fhrp_group_assignments": "fhrp_group_assignment",
     "front_ports": "front_port",
+    "front-ports": "front_port",
     "front_port_templates": "front_port_template",
     "journal_entries": "journal_entry",
     "interfaces": "interface",
@@ -378,6 +383,7 @@ ENDPOINT_NAME_MAPPING = {
     "power_outlet_templates": "power_outlet_template",
     "power_panels": "power_panel",
     "power_ports": "power_port",
+    "power-ports": "power_port",
     "power_port_templates": "power_port_template",
     "prefixes": "prefix",
     "providers": "provider",

EDIT: Upon some further digging it looks like the root cause may be due to the differences in the endpoint mapping between the lookup module's netbox_endpoint_map that defines endpoint names with dashes and the mapping in module utils that defines endpoint names with underscores.

The reason that dcim.rearports works correctly is that it is already defined in API_APPS_ENDPOINTS and ENDPOINT_NAME_MAPPING with both dashes and underscores.

Possible solutions could be to either take the naive approach that I took above and define endpoints with both dashes and underscores in netbox_utils.py but that may not scale well. Alternatively another approach could be substituting dashes for underscores in NetboxModule._find_app() and before indexing into ENDPOINT_NAME_MAPPING