nautobot / nautobot-app-golden-config

Golden Configuration App for Nautobot.
https://docs.nautobot.com/projects/golden-config/en/latest/
Other
101 stars 57 forks source link

Support for non UTF-8 characters in the configuration #802

Open bminnix opened 2 months ago

bminnix commented 2 months ago

Environment

Proposed Functionality

Allow non UTF-8 characters to be read into the configuration when comparing the backup and intended values.

Use Case

Currently the diff function is setting encoding to utf-8

nautobot-golden-config/nornir_plays/config_compliance.py

def diff_files(backup_file, intended_file):
    """Utility function to provide `Unix Diff` between two files."""
    with open(backup_file, encoding="utf-8") as file:
        backup = file.readlines()
    with open(intended_file, encoding="utf-8") as file:
        intended = file.readlines()

We've ran into a case where a legit, non UTF-8 encoded character is in the device running config output and causes a UnicodeDecodeError.

! NAME: "<ifacename>", DESCR: "<description>"
! PID: SFP-10G-SR        , VID: V03, SN: xxxxxxxx�xx

In this specific situation, the value is being caused by an SFP module that has a non-standard S/N value, yet it is the assigned valid value read from the module. The enforcement of UTF-8 is causing this to fail while being read into the function.

Since these characters are permitted into the operating system, there may be other cases for other values where they are present and also legit (but I can't think of any).