networktocode / network-importer

The network importer is a tool/library to analyze and/or synchronize an existing network with a Network Source of Truth (SOT), it's designed to be idempotent and by default it's only showing the difference between the running network and the remote SOT.
Other
170 stars 41 forks source link

invalid DiffElement - just fresh install of network-importer-3.0.3 #256

Open hikarikumo opened 2 years ago

hikarikumo commented 2 years ago

Environment

Configuration

[main] import_ips = true import_prefixes = true import_intf_status = false import_vlans = "config" import_cabling = "config" excluded_platforms_cabling = ["cisco_asa"] nbr_workers = 25 configs_directory = "configs" backend = "nautobot" [inventory] supported_platforms = [ "cisco_ios", "cisco_nxos", "arista_eos" ] [inventory.settings] address = "" token = "" verify_ssl = false [batfish] address= "localhost" network_name = "network-importer" # Alternative Env Variable : BATFISH_NETWORK_NAME port_v1 = 9997 # Alternative Env Variable : BATFISH_PORT_V1 port_v2 = 9996 # Alternative Env Variable : BATFISH_PORT_V2 use_ssl = false # Alternative Env Variable : BATFISH_USE_SSL [network] login = "cisco" # Alternative Env Variable : NETWORK_DEVICE_LOGIN password = "cisco" # Alternative Env Variable : NETWORK_DEVICE_PWD enable = true # Alternative Env Variable : NETWORK_DEVICE_ENABLE fqdns = [ ] [network.netmiko_extras] [network.napalm_extras] [drivers.mapping] default = "network_importer.drivers.default" cisco_nxos = "network_importer.drivers.cisco_default" cisco_ios = "network_importer.drivers.cisco_default" cisco_xr = "network_importer.drivers.cisco_default" juniper_junos = "network_importer.drivers.juniper_junos" arista_eos = "network_importer.drivers.arista_eos" [logs] level = "info" # "debug", "info", "warning" performance_log = false performance_log_directory = "performance_logs"

Steps to Reproduce

  1. install network-importer
  2. created configuration
  3. run network-importer check --update-configs

Expected Behavior

It was expected to receive valid output, however, I had received the errors instead

Observed Behavior

Traceback (network-importer-venv) ✔ ~/git/network-importer-test [main|✔] 15:02 $ network-importer check 2022-02-07 15:02:45,333 - network-importer - INFO - Import SOT Model 2022-02-07 15:02:47,255 - network-importer - INFO - Import Network Model Traceback (most recent call last): File "/home/hikari/network-importer-venv/bin/network-importer", line 8, in sys.exit(main()) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 829, in call return self.main(args, kwargs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(args, **kwargs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/cli.py", line 149, in check diff = ni.diff() File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/main.py", line 146, in diff return self.sot.diff_from(self.network, diff_class=NetworkImporterDiff) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/init.py", line 594, in diff_from return differ.calculate_diffs() File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 87, in calculate_diffs diff_elements = self.diff_object_list( File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 132, in diff_object_list if diff_element: File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/diff.py", line 219, in len for child in self.get_children(): File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/diff.py", line 306, in get_children yield from self.child_diff.get_children() File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/diff.py", line 95, in get_children yield from order_method(self.children[group]) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/diff.py", line 53, in order_children_interface raise Exception("invalid DiffElement") Exception: invalid DiffElement

nstamoul commented 2 years ago

I 'll second that, I am facing the exact same situation.

Same python version, same network importer version.

[edit]

A quick and dirty fix that seems to work for me is the following

``` action = child.action if not action: action = "update" else: action = child.action.value ```
hikarikumo commented 2 years ago

hello @nstamoul Could you please guide me on what exactly has to be changed? I have changed it into the following ' def order_children_interface(cls, children): """Return the interface children ordered order.""" intfs_lags = defaultdict(list) intfs_regs = defaultdict(list) intfs_lag_members = defaultdict(list)

    for child_name, child in children.items():
        action = child.action
        if not action:
            action = "update"
        if action == "delete":
            if "is_lag" in child.dest_attrs and child.dest_attrs["is_lag"]:
                intfs_lags[action].append(child_name)
            elif "is_lag_member" in child.dest_attrs and child.dest_attrs["is_lag_member"]:
                intfs_lag_members[action].append(child_name)
            else:
                intfs_regs[action].append(child_name)
        elif action in ["update", "create"]:
            if "is_lag" in child.source_attrs and child.source_attrs["is_lag"]:
                intfs_lags[action].append(child_name)
            elif "is_lag_member" in child.source_attrs and child.source_attrs["is_lag_member"]:
                intfs_lag_members[action].append(child_name)
            else:
                intfs_regs[action].append(child_name)
        else:
            action = child.action.value
            # raise Exception("invalid DiffElement")

' The check now runs fine, however apply fails: Check $ network-importer check --update-configs 2022-02-15 08:33:22,070 - network-importer - INFO - Updating configuration from devices .. 2022-02-15 08:33:23,055 - network-importer - INFO - R1 | Latest config file already present ... 2022-02-15 08:33:23,132 - network-importer - INFO - R3 | Latest config file already present ... 2022-02-15 08:33:23,133 - network-importer - INFO - Import SOT Model 2022-02-15 08:33:23,971 - network-importer - INFO - Import Network Model site site: home vlan vlan: home5 MISSING in Network vlan: home6 MISSING in Network prefix prefix: home172.16.244.0/24 MISSING in Nautobot prefix: home192.168.100.0/24 MISSING in Nautobot device device: R1 interface interface: Ethernet0/1 MISSING in Nautobot interface: Ethernet0/2 MISSING in Nautobot interface: Ethernet0/0 ip_address ip_address: R1Ethernet0/0172.16.244.101/24 MISSING in Nautobot interface: Ethernet0/3 MISSING in Nautobot device: R3 interface interface: Ethernet0/3 MISSING in Nautobot interface: Ethernet0/1 MISSING in Nautobot interface: Ethernet0/2 MISSING in Nautobot 2022-02-15 08:33:26,586 - network-importer - INFO - Execution finished, processed 2 device(s)

Apply $ network-importer apply 2022-02-15 08:33:56,032 - network-importer - INFO - Import SOT Model 2022-02-15 08:33:56,710 - network-importer - INFO - Import Network Model Traceback (most recent call last): File "/home/hikari/network-importer-venv/bin/network-importer", line 8, in sys.exit(main()) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 829, in call return self.main(args, kwargs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(args, **kwargs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/cli.py", line 108, in apply ni.sync() File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/main.py", line 142, in sync self.sot.sync_from(self.network, diff_class=NetworkImporterDiff) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/init.py", line 525, in sync_from result = syncer.perform_sync() File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 326, in perform_sync changed |= self.sync_diff_element(element) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 390, in sync_diff_element changed |= self.sync_diff_element(child, parent_model=dst_model) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 366, in sync_diff_element changed, modified_model = self.sync_model(src_model=src_model, dst_model=dst_model, ids=ids, attrs=attrs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 415, in sync_model dst_model = self.model_class.create(diffsync=self.dst_diffsync, ids=ids, attrs=attrs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/adapters/nautobot_api/models.py", line 436, in create LOGGER.info("Created Prefix %s (%s) in Nautobot", prefix.prefix, prefix.id) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/pynautobot/core/response.py", line 192, in getattr raise AttributeError('object has no attribute "{}"'.format(k)) AttributeError: object has no attribute "prefix"

nstamoul commented 2 years ago

Hi @hikarikumo , that seems to be a different issue, so I am afraid I can't really help you there.

You could always try troubleshooting the problem, starting by finding out what kind of object k is.

nstamoul commented 2 years ago

Hi @hikarikumo , that seems to be a different issue, so I am afraid I can't really help you there.

You could always try troubleshooting the problem, starting by finding out what kind of object k is.

glennmatthews commented 2 years ago

This appears to be due to a breaking change in diffsync 1.4.x (https://github.com/networktocode/diffsync/pull/90). If that's the case, doing a pip install diffsync==1.3.0 should resolve this issue until this library can be updated to account for that change.