When using the validate / compliance_report function, the list validation is not done properly.
For a given "item" in a validation file, if this item is specified as a list, then the _compare_getter_list() method is called. This method used compare() for each element in the list and if the elements are strings, then the re.search() method is used.
The issue is by doing that, false positive can be found as the re.search() method only look for a partial match.
For example, if we have a validation file checking for "Vlan5" interface in the interface_list item, and that the device sends "Vlan5554", it will be considered as a positive result even though it's not.
A quick "fix" I can think of is the use of re.fullmatch() instead of re.search(). Or introduce a specific syntax in the validation file to precise if we're talking about pattern or "complete strings".
Description of Issue/Question
When using the validate / compliance_report function, the list validation is not done properly. For a given "item" in a validation file, if this item is specified as a list, then the _compare_getter_list() method is called. This method used compare() for each element in the list and if the elements are strings, then the re.search() method is used.
The issue is by doing that, false positive can be found as the re.search() method only look for a partial match. For example, if we have a validation file checking for "Vlan5" interface in the interface_list item, and that the device sends "Vlan5554", it will be considered as a positive result even though it's not.
A quick "fix" I can think of is the use of re.fullmatch() instead of re.search(). Or introduce a specific syntax in the validation file to precise if we're talking about pattern or "complete strings".
Did you follow the steps from https://github.com/napalm-automation/napalm#faq
(Place an
x
between the square brackets where applicable)Setup
napalm version
(Paste verbatim output from
pip freeze | grep napalm
between quotes below)Network operating system version
(Paste verbatim output from
show version
- or equivalent - between quotes below)Steps to Reproduce the Issue