expand doesn't work with embedded groups

[mouseroyal]

When embedded groups are used within the top group, it can affect how the functions work that are defined for top group.

Example:

interface GigabitEthernet1/1 description to core-1 mtu 9100 switchport mode trunk !

interface {{ name }} description {{ config.description | ORPHRASE }} mtu {{ config.mtu | to_int }} switchport mode {{ mode }} ! {{_end_}}

This generates following output, where the match variables config.description and config.mut are correctly expanded: [ [ { "interfaces": { "config": { "description": "to core-1", "mtu": 9100 }, "name": "GigabitEthernet1/1", "switched-vlans": { "mode": "trunk" } } } ] ]

However, if the input data is changed to following (note: "mtu 9100" comes after "switchport mode trunk")

interface GigabitEthernet1/1 description to core-1 switchport mode trunk mtu 9100 !

It generates following, where config.description is expanded correctly, but config.mtu is not: [ [ { "interfaces": { "config": { "description": "to core-1" }, "config.mtu": 9100, "name": "GigabitEthernet1/1", "switched-vlans": { "mode": "trunk" } } } ] ]

Expected result: I am expecting the functions configured for top group to be executed after the ENTIRE group has done matching. Both config.description and config.mtu should get expanded, even if there is another line in the middle which matched a subgroup "switched-vlans"

Actual result: It looks that whenever there is a subgroup match, it will trigger some kind of a context switch, where the top group functions got executed (i.e., config.description got expanded), but when the subgroup matching is done and it comes back to remainder of top group, the context is not switched back properly, and NONE of the defined functions will get executed (config.mtu is not getting expanded). This includes but is not limited to expand() and macro().

Is this a bug or a design limitation?

[dmulyalin]

Hi, thank you for raising that issue, would have a better chance to look at it if you can provide data/templates with expected/actual results wrapped into code blocks using "`" symbols, as of now I am having hard time decoding template and data contents.

Answering - It looks that whenever there is a subgroup match, it will trigger some kind of a context switch - yeas, this is the way how TTP works by design, this is to simplify processing logic, each and every group, even nested/child groups handled independently one from another, parent's groups functions does not have effect on child groups results because they do not see them.

If existing TTP capabilities does not fully meet your needs, suggest to produce some structured data using as simple as possible template and do post-processing of resulted structure in Python transforming them into desired structure.

[mouseroyal]

Sorry about the un-readability, here is the code snippet to reproduce the issue. This is working code:

<input load="text">
interface GigabitEthernet1/1
   description to core-1
   mtu 9100
   switchport mode trunk
!
</input>

<group name="interfaces.interface*" expand="" >
interface {{ name }}
   description {{ config.description | ORPHRASE }}
   mtu {{ config.mtu | to_int }}
   <group name="switched-vlan">
   switchport mode {{ mode }}
   </group>
! {{_end_}}
</group>

<output returner="terminal" format="json" />

And if you run it with "ttp -t" it will show following output:

[
    {
        "interfaces": {
            "interface": [
                {
                    "config": {
                        "description": "to core-1",
                        "mtu": 9100
                    },
                    "name": "GigabitEthernet1/1",
                    "switched-vlan": {
                        "mode": "trunk"
                    }
                }
            ]
        }
    }
]

As you can see, config.description and config.mtu are properly expanded.

Here is the non-working code, and the only change is for the input part, where "mtu 9100" is moved below "switchport mode trunk":

<input load="text">
interface GigabitEthernet1/1
   description to core-1
   switchport mode trunk
   mtu 9100
!
</input>

<group name="interfaces.interface*" expand="">
interface {{ name }}
   description {{ config.description | ORPHRASE }}
   mtu {{ config.mtu | to_int }}
   <group name="switched-vlan">
   switchport mode {{ mode }}
   </group>
! {{_end_}}
</group>

<output returner="terminal" format="json" />

And this is output:

[
    {
        "interfaces": {
            "interface": [
                {
                    "config": {
                        "description": "to core-1"
                    },
                    "config.mtu": 9100,
                    "name": "GigabitEthernet1/1",
                    "switched-vlan": {
                        "mode": "trunk"
                    }
                }
            ]
        }
    }
]

You can see that config.mtu is no longer expanded, because the subgroup "switched-vlan" matched line "switchport mode trunk" first, and when matching exited the subgroup and resumed in the main group, it looks that it forgot about the "expand" flag

[dmulyalin]

Thank you for additional information, looks like a bug, will investigate it.

[dmulyalin]

@mouseroyal that should be fixed starting with commit 48f1dde, you can wait till next release to test it or install from master branch if want to give it a try before that:

python3 -m pip install git+https://github.com/dmulyalin/ttp

[dmulyalin]

FYI - released TTP 0.8.4 that includes fix for tis issue.

[mouseroyal]

FYI - released TTP 0.8.4 that includes fix for tis issue.

Thanks for the prompt fix!