XML Compliance check seems not working with Palo Alto

baldy2811 commented 3 weeks ago

Environment

Python version: 3.11
Nautobot version: 2.3.1
nautobot-golden-config version: 2.1.1

I use XML Format for Palo Alto compliance Checks. Current Config and intended config are the same but is shows non-complaint

Steps to Reproduce

Create Compliance Rule in XML Format with XPath: /response/result/config/devices/entry/network/interface/loopback
Run the Compliace Check to valdidate if a device is complaint

Intendend config:

<response>
  <result>
    <config detail-version="10.2.10" version="10.2.0" urldb="paloaltonetworks">
      <devices>
        <entry name="localhost.localdomain">
          <network>
            <interface>
              <loopback>
                <units>
                  <entry name="loopback.103">
                    <adjust-tcp-mss>
                      <enable>no</enable>
                    </adjust-tcp-mss>
                    <ip>
                      <entry name="172.22.1.1/32"/>
                    </ip>
                    <interface-management-profile>PING-AND-SSH</interface-management-profile>
                    <comment>Loopback for test SEA</comment>
                  </entry>
                </units>
              </loopback>
            </interface>
          </network>
        </entry>
      </devices>
    </config>
  </result>
</response>

Actual Config:

<response>
  <result>
    <config detail-version="10.2.10" version="10.2.0" urldb="paloaltonetworks">
      <devices>
        <entry name="localhost.localdomain">
          <network>
            <interface>
              <loopback>
                <units>
                  <entry name="loopback.103">
                    <adjust-tcp-mss>
                      <enable>no</enable>
                    </adjust-tcp-mss>
                    <ip>
                      <entry name="172.22.1.1/32"/>
                    </ip>
                    <interface-management-profile>PING-AND-SSH</interface-management-profile>
                    <comment>Loopback for test SEA</comment>
                  </entry>
                </units>
              </loopback>
            </interface>
          </network>
        </entry>
      </devices>
    </config>
  </result>
</response>

Status is: Status - Non-Compliant

I can check with every parameter within XPath and everything is non-compliant. We try to use it with Palo Alto backup XML files.

itdependsnetworks commented 3 weeks ago

@jmpettit can you take a look?

jmpettit commented 3 weeks ago

@baldy2811 can you run this script on your actual/intended snippets (the ones attached to the issue show identical to me)

from xmldiff import main

intended = """
<response>
  <result>
    <config detail-version="10.2.10" version="10.2.0" urldb="paloaltonetworks">
      <devices>
        <entry name="localhost.localdomain">
          <network>
            <interface>
              <loopback>
                <units>
                  <entry name="loopback.103">
                    <adjust-tcp-mss>
                      <enable>no</enable>
                    </adjust-tcp-mss>
                    <ip>
                      <entry name="172.22.1.1/32"/>
                    </ip>
                    <interface-management-profile>PING-AND-SSH</interface-management-profile>
                    <comment>Loopback for test SEA</comment>
                  </entry>
                </units>
              </loopback>
            </interface>
          </network>
        </entry>
      </devices>
    </config>
  </result>
</response>
"""

actual = """
<response>
  <result>
    <config detail-version="10.2.10" version="10.2.0" urldb="paloaltonetworks">
      <devices>
        <entry name="localhost.localdomain">
          <network>
            <interface>
              <loopback>
                <units>
                  <entry name="loopback.103">
                    <adjust-tcp-mss>
                      <enable>no</enable>
                    </adjust-tcp-mss>
                    <ip>
                      <entry name="172.22.1.1/32"/>
                    </ip>
                    <interface-management-profile>PING-AND-SSH</interface-management-profile>
                    <comment>Loopback for test SEA</comment>
                  </entry>
                </units>
              </loopback>
            </interface>
          </network>
        </entry>
      </devices>
    </config>
  </result>
</response>
"""

diff_options = {
    "F": 0.1,
    "fast_match": True,
}

missing = main.diff_texts(actual, intended, diff_options=diff_options)
extra = main.diff_texts(intended, actual, diff_options=diff_options)

print(missing)
print(extra)

baldy2811 commented 3 weeks ago

@jmpettit thats exactly the point. Its identical but it shows non-compliant. Which python version are you on?

jmpettit commented 3 weeks ago

@jmpettit thats exactly the point. Its identical but it shows non-compliant. Which python version are you on?

ok interesting, under the hood this is using xmldiff - I assumed my _normalize_diff function was causing issues here.

https://github.com/nautobot/nautobot-app-golden-config/blob/2e39e7d50793429d1d4fad140e2b3742d86eafc2/nautobot_golden_config/models.py#L122

I ran that script from nautobot-server shell_plus Python 3.11.9 in my dev instance, I was looking to see if extra/missing had something in them in your case, which is what causes the non-compliant status. I'll finish setting up my dev instance and test a bit further with your exact config snippets.

baldy2811 commented 2 weeks ago

@jmpettit i run the script you posted above and got the following output:

$ python test.py
[]
[]

jmpettit commented 2 weeks ago

@jmpettit i run the script you posted above and got the following output:
$ python test.py
[]
[]

Thanks, could you also show the detail view by hitting the "test" link in your Configuration Compliance view, example: https://demo.nautobot.com/plugins/golden-config/config-compliance/0455fb5b-27dd-4823-8e1b-55e2f046b241/?tab=main

we are wondering if this is related to #800

baldy2811 commented 2 weeks ago

It still non-compliant

itdependsnetworks commented 2 weeks ago

Met with @jmpettit we see where the challenges arise from. We spoke about an approach, will get back to you with some more formalized thoughts over the next few days.

jmpettit commented 2 weeks ago

@baldy2811 can you run this via the ORM (nautobot-server shell_plus) and then run compliance against the device again, thanks!

obj = ConfigCompliance.objects.get(device__name="SDWAN-HUB-LAB-01")
obj.delete()

baldy2811 commented 2 weeks ago

@jmpettit

>>> obj = ConfigCompliance.objects.get(device__name="SDWAN-HUB-LAB-01")
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/opt/nautobot/.local/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nautobot/.local/lib/python3.11/site-packages/django/db/models/query.py", line 640, in get
    raise self.model.MultipleObjectsReturned(
nautobot_golden_config.models.ConfigCompliance.MultipleObjectsReturned: get() returned more than one ConfigCompliance -- it returned 3!

>>> obj.delete()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
NameError: name 'obj' is not defined

i did this on all docker nodes and get everywhere the same error.

baldy2811 commented 2 weeks ago

Crazy,

after deleting and re-adding the rules it seems working:

itdependsnetworks commented 1 week ago

~Should be fixed in 2.1.2, let me know if there are any other issues~ nvm

jmpettit commented 1 week ago

fixed in 2.1.2

nautobot / nautobot-app-golden-config

XML Compliance check seems not working with Palo Alto #795

Environment

Steps to Reproduce