networktocode / netutils

Python library that is a collection of functions and objects for common network automation tasks.
https://netutils.readthedocs.io/
Other
214 stars 48 forks source link

regex_search parameter ordering is a problem when used from Jinja #523

Closed MeganerdDev closed 4 months ago

MeganerdDev commented 4 months ago

Environment

Expected Behavior

The expectation is that the netutils.regex.regex_search will return a match, same as re.regex_search

Observed Behavior

When using the netutils.regex.regex_search filter from the provided Jinja example the regular expression is not finding a match, however re.regex_search is working directly. At closer inspection, I can see the issue is the order of the string and regex parameters are flipped. When flipping the order to match re.regex_search everything works as expected.

From python we can specify the arguments directly so this is not an issue there. However when trying to use this from Jinja we do run in to this issue.

Steps to Reproduce

  1. Reproducing with concept script below
import re
from jinja2 import Environment, DictLoader
import typing as t

def _match_object(match: t.Optional[t.Match[str]]) -> t.Union[t.List[str], str, None]:
    """Helper method to better 'serialize' a re.Match object."""
    if not match:
        return None
    if match.groups():
        results = []
        for group in match.groups():
            results.append(group)
        return results
    return str(match.group())

def netutils_regex_search(pattern: str, string: str) -> t.Union[t.List[str], str, None]:
#def netutils_regex_search(string: str, pattern: str) -> t.Union[t.List[str], str, None]: # flipped parameter ordering
    r"""Given a regex pattern and string, return `None` if there is no matching `re.Match.groups()` if using capture groups or regex match via `re.Match.group()`.

    The main purpose of this function is provide a Jinja2 filter as this is simply a wrapper around `re.search`.

    Args:
        pattern: Regex string to match against.
        string: String to check against.

    Returns:
        List of matches, match, or None no match found.

    Examples:
        >>> from netutils.regex import regex_search
        >>> print("South Carolina" if regex_search(".+SC.+\d\d", "USSCAMS07") else "Not South Carolina")
        South Carolina
        >>>
        >>> match = regex_search("^([A-Z]{2})([A-Z]{2})([A-Z]{3})(\d*)", "USSCAMS07")
        >>> match[0]
        'US'
        >>> match[1]
        'SC'
        >>> match[2]
        'AMS'
        >>> match[3]
        '07'

    """
    return _match_object(re.search(pattern, string))

def regex_search(value, pattern):
    return re.search(pattern, value) is not None

env = Environment(loader=DictLoader({'template': '''
{% for interface in interfaces %}
    {% if interface.name | regex_search('GigabitEthernet\d\/0\/\d+') and interface.mode == 'ACCESS' %}
        match found.
    {% endif %}
{% endfor %}
'''}))

data = {
    "interfaces": [
        {
            "mode": "ACCESS",
            "name": "GigabitEthernet1/0/1",
        },
        {
            "mode": "ACCESS",
            "name": "GigabitEthernet1/0/2",
        }
    ]
}

# test using normal re.search
env.filters["regex_search"] = regex_search
template = env.get_template("template")
print(template.render(interfaces=data["interfaces"]))

# test using netutils.regex_search
env.filters["regex_search"] = netutils_regex_search
template = env.get_template("template")
print(template.render(interfaces=data["interfaces"]))
MeganerdDev commented 4 months ago

To provide context from testing on this:

tested: passing multiple parameters is not working

env = Environment(loader=DictLoader({'template': '''
{% for interface in interfaces %}
    {% if regex_search(interface.name, 'GigabitEthernet\d\/0\/\d+') and interface.mode == 'ACCESS' %}
        match found.
    {% endif %}
{% endfor %}
'''}))

tested: using a Jinja macro is a working mitigation

env = Environment(loader=DictLoader({'template': '''
{% macro regex_search_macro(value, pattern) %}
    {{ value | regex_search(pattern) }}
{% endmacro %}

{% for interface in interfaces %}
    {% if regex_search_macro(interface.name, 'GigabitEthernet\d\/0\/\d+') and interface.mode == 'ACCESS' %}
        match found.
    {% endif %}
{% endfor %}
'''}))
itdependsnetworks commented 4 months ago

This needs to be flipped:

    {% if interface.name | regex_search('GigabitEthernet\d\/0\/\d+') and interface.mode == 'ACCESS' %}

becomes

    {% if 'GigabitEthernet\\d/0/\\d+' | regex_search(interface.name) and interface.mode == 'ACCESS' %}
itdependsnetworks commented 4 months ago

pretty confident that is the issue, but feel free to open back up if need to.