dmulyalin / ttp

Template Text Parser
MIT License
350 stars 34 forks source link

Ability to make a flat structure of the output #39

Closed knpshkn closed 3 years ago

knpshkn commented 3 years ago

Hello! I just started to use this library and it looks very handy I am trying to make the output structure as flat as possible, for example I have an input

CAM Utilization for ASIC# 0                      Max            Used
                                             Masks/Values    Masks/values

 Unicast mac addresses:                        400/3200         50/300   
 IPv4 IGMP groups + multicast routes:          152/1216          7/27    
 IPv4 unicast directly-connected routes:       400/3200         50/300   
 IPv4 unicast indirectly-connected routes:    1040/8320         58/377   
 IPv4 policy based routing aces:               384/512           1/2     
 IPv4 qos aces:                                768/768         390/390   
 IPv4 security aces:                          1024/1024         39/39    

Note: Allocation of TCAM entries per feature uses
a complex algorithm. The above information is meant
to provide an abstract view of the current TCAM utilization

CAM Utilization for ASIC# 1                      Max            Used
                                             Masks/Values    Masks/values

 Unicast mac addresses:                        400/3200         50/300   
 IPv4 IGMP groups + multicast routes:          152/1216          7/27    
 IPv4 unicast directly-connected routes:       400/3200         50/300   
 IPv4 unicast indirectly-connected routes:    1040/8320         58/377   
 IPv4 policy based routing aces:               384/512           1/2     
 IPv4 qos aces:                                768/768         390/390   
 IPv4 security aces:                          1024/1024         39/39    

Note: Allocation of TCAM entries per feature uses
a complex algorithm. The above information is meant
to provide an abstract view of the current TCAM utilization

but the best I could get out of this doesn't quite suit my needs

template:
<group method="table" del="max_name, used_name, max_mask, used_mask">
CAM Utilization for {{ asic | replaceall('#') | ORPHRASE }} {{ max_name }} {{ used_name }}
 {{ cam | ORPHRASE }}: {{ max_mask }}/{{ max }} {{ used_mask }}/{{ used }}
</group>

output:
[
    [
        [
            {
                "asic": "ASIC 0"
            },
            {
                "cam": "Unicast mac addresses",
                "max": "3200",
                "used": "300"
            },
            {
                "cam": "IPv4 IGMP groups + multicast routes",
                "max": "1216",
                "used": "27"
            },
            {
                "cam": "IPv4 unicast directly-connected routes",
                "max": "3200",
                "used": "300"
            },
            {
                "cam": "IPv4 unicast indirectly-connected routes",
                "max": "8320",
                "used": "377"
            },
            {
                "cam": "IPv4 policy based routing aces",
                "max": "512",
                "used": "2"
            },
            {
                "cam": "IPv4 qos aces",
                "max": "768",
                "used": "390"
            },
            {
                "cam": "IPv4 security aces",
                "max": "1024",
                "used": "39"
            },
            {
                "asic": "ASIC 1"
            },
            {
                "cam": "Unicast mac addresses",
                "max": "3200",
                "used": "300"
            },
            {
                "cam": "IPv4 IGMP groups + multicast routes",
                "max": "1216",
                "used": "27"
            },
            {
                "cam": "IPv4 unicast directly-connected routes",
                "max": "3200",
                "used": "300"
            },
            {
                "cam": "IPv4 unicast indirectly-connected routes",
                "max": "8320",
                "used": "377"
            },
            {
                "cam": "IPv4 policy based routing aces",
                "max": "512",
                "used": "2"
            },
            {
                "cam": "IPv4 qos aces",
                "max": "768",
                "used": "390"
            },
            {
                "cam": "IPv4 security aces",
                "max": "1024",
                "used": "39"
            }
        ]
    ]
]
or
template:
<template results="per_template">
<group method="table" del="max_name, used_name">
CAM Utilization for {{ asic | replaceall('#') | ORPHRASE }} {{ max_name }} {{ used_name }}
 <group name="_" del="max_mask, used_mask">
 {{ cam | ORPHRASE }}: {{ max_mask }}/{{ max }} {{ used_mask }}/{{ used }}
 </group>
</group>
</template>

output:
[
    [
        {
            "asic": "ASIC 0",
            "cam": "IPv4 security aces",
            "max": "1024",
            "used": "39"
        },
        {
            "asic": "ASIC 1",
            "cam": "IPv4 security aces",
            "max": "1024",
            "used": "39"
        }
    ]
]

maybe there is a way to "copy" the variable inside each object or you can advise me something. I would like to get something like this

[
    [
        [
            {
                "asic": "ASIC 0"
                "cam": "Unicast mac addresses",
                "max": "3200",
                "used": "300"
            },
            {
                "asic": "ASIC 0"
                "cam": "IPv4 IGMP groups + multicast routes",
                "max": "1216",
                "used": "27"
            },
...
            {
                "asic": "ASIC 1"
                "cam": "IPv4 qos aces",
                "max": "768",
                "used": "390"
            },
            {
                "asic": "ASIC 1"
                "cam": "IPv4 security aces",
                "max": "1024",
                "used": "39"
            }
        ]
    ]
]
dmulyalin commented 3 years ago

Hi,

Think this template produces what you want:

from ttp import ttp
import pprint

data = """
CAM Utilization for ASIC# 0                      Max            Used
                                             Masks/Values    Masks/values

 Unicast mac addresses:                        400/3200         50/300   
 IPv4 IGMP groups + multicast routes:          152/1216          7/27    
 IPv4 unicast directly-connected routes:       400/3200         50/300   
 IPv4 unicast indirectly-connected routes:    1040/8320         58/377   
 IPv4 policy based routing aces:               384/512           1/2     
 IPv4 qos aces:                                768/768         390/390   
 IPv4 security aces:                          1024/1024         39/39    

Note: Allocation of TCAM entries per feature uses
a complex algorithm. The above information is meant
to provide an abstract view of the current TCAM utilization

CAM Utilization for ASIC# 1                      Max            Used
                                             Masks/Values    Masks/values

 Unicast mac addresses:                        400/3200         50/300   
 IPv4 IGMP groups + multicast routes:          152/1216          7/27    
 IPv4 unicast directly-connected routes:       400/3200         50/300   
 IPv4 unicast indirectly-connected routes:    1040/8320         58/377   
 IPv4 policy based routing aces:               384/512           1/2     
 IPv4 qos aces:                                768/768         390/390   
 IPv4 security aces:                          1024/1024         39/39    

Note: Allocation of TCAM entries per feature uses
a complex algorithm. The above information is meant
to provide an abstract view of the current TCAM utilization
"""

template = """
<group name="CAM" chain="record(asic, asic_name) | void()">
CAM Utilization for {{ asic | replaceall('#') | ORPHRASE }} {{ max_name }} {{ used_name }}
 <group name="/" set="asic_name, asic">
 {{ cam | ORPHRASE }}: {{ ignore }}/{{ max }} {{ ignore }}/{{ used }}
 </group>
</group>
"""

parser = ttp(data, template)
parser.parse()
result = parser.result(structure="flat_list")
pprint.pprint(result, width=150)

# prints:
# [{'asic': 'ASIC 0', 'cam': 'Unicast mac addresses', 'max': '3200', 'used': '300'},
#  {'asic': 'ASIC 0', 'cam': 'IPv4 IGMP groups + multicast routes', 'max': '1216', 'used': '27'},
#  {'asic': 'ASIC 0', 'cam': 'IPv4 unicast directly-connected routes', 'max': '3200', 'used': '300'},
#  {'asic': 'ASIC 0', 'cam': 'IPv4 unicast indirectly-connected routes', 'max': '8320', 'used': '377'},
#  {'asic': 'ASIC 0', 'cam': 'IPv4 policy based routing aces', 'max': '512', 'used': '2'},
#  {'asic': 'ASIC 0', 'cam': 'IPv4 qos aces', 'max': '768', 'used': '390'},
#  {'asic': 'ASIC 0', 'cam': 'IPv4 security aces', 'max': '1024', 'used': '39'},
#  {'asic': 'ASIC 1', 'cam': 'Unicast mac addresses', 'max': '3200', 'used': '300'},
#  {'asic': 'ASIC 1', 'cam': 'IPv4 IGMP groups + multicast routes', 'max': '1216', 'used': '27'},
#  {'asic': 'ASIC 1', 'cam': 'IPv4 unicast directly-connected routes', 'max': '3200', 'used': '300'},
#  {'asic': 'ASIC 1', 'cam': 'IPv4 unicast indirectly-connected routes', 'max': '8320', 'used': '377'},
#  {'asic': 'ASIC 1', 'cam': 'IPv4 policy based routing aces', 'max': '512', 'used': '2'},
#  {'asic': 'ASIC 1', 'cam': 'IPv4 qos aces', 'max': '768', 'used': '390'},
#  {'asic': 'ASIC 1', 'cam': 'IPv4 security aces', 'max': '1024', 'used': '39'}]

How it works

chain="record(asic, asic_name) | void()" - have a look at group functions docs for description, but record stores parent group asic match variable value in cache in variable asic_name, for each next name="CAM" group, that value gets updated set="asic_name, asic" - retrieves asic_name value from variables cache and assigns it to asic variable within this group result name="/" - uses absolute path feature to tell that result should be saved at the very top {{ ignore }}/{{ max }} {{ ignore }}/{{ used }} - ignore statement help to ignore match results structure="flat_list" - api reference gives good explaination

knpshkn commented 3 years ago

Thank you, works great! A very interesting solution