CiscoTestAutomation / genieparser

sub-component of Genie that parse the device output into structured datastructure
Apache License 2.0
243 stars 370 forks source link

[IOS-XR] Parser for "show isis statistics" does not work #853

Open ChristopherJHart opened 2 weeks ago

ChristopherJHart commented 2 weeks ago

Sample CLI output:

RP/0/RSP0/CPU0:R5#show isis statistics
Wed Apr 10 10:58:18.269 EDT

IS-IS 64512 statistics:
    Fast PSNP cache (hits/tries): 91099/146551
    Fast CSNP cache (hits/tries): 5/30
    Fast CSNP cache updates: 50
    LSP checksum errors received: 0
    LSP Dropped: 0
    SNP Dropped: 0
    UPD Max Queue size: 8
    UPD Queue size: 0
    Average transmit times and rates:
      Hello:          0 us          2/s
      CSNP:           0 us          0/s
      PSNP:           0 us          0/s
      LSP:            0 us          0/s
    Average process times and rates:
      Hello:          0 us          2/s
      CSNP:           0 us          0/s
      PSNP:           0 us          0/s
      LSP:            0 us          0/s
    Level-2:
      LSPs sourced (new/refresh): 50/16892
      IPv4 Unicast
        Total SPF calculations     : 7526
        Full SPF calculations      : 7508
        ISPF calculations          : 0
        Next Hop Calculations      : 11
        Partial Route Calculations : 7
        Periodic SPF calculations  : 7382
  Interface Loopback0:
  Interface TenGigE0/0/0/0.10:
    PTP Hellos (sent/rcvd)    : 760149/760205
    LSP Retransmissions       : 0
    LSP Peak Rate (out/in)    : 2/3
    Level-2 LSPs (sent/rcvd)  : 49222/53198
    Level-2 CSNPs (sent/rcvd) : 1/1
    Level-2 PSNPs (sent/rcvd) : 52991/49049
    Level-2 LSP Flooding Duplicates     : 7
    Level-2 LSPs Arrival Time Throttled : 0
  Interface TenGigE0/0/0/0.20:
    PTP Hellos (sent/rcvd)    : 760305/760270
    LSP Retransmissions       : 1
    LSP Peak Rate (out/in)    : 1/8
    Level-2 LSPs (sent/rcvd)  : 4/12
    Level-2 CSNPs (sent/rcvd) : 1/1
    Level-2 PSNPs (sent/rcvd) : 4/3
    Level-2 LSP Flooding Duplicates     : 22912
    Level-2 LSPs Arrival Time Throttled : 0
  Interface TenGigE0/0/0/1.10:
    PTP Hellos (sent/rcvd)    : 760320/760317
    LSP Retransmissions       : 0
    LSP Peak Rate (out/in)    : 0/0
    Level-2 LSPs (sent/rcvd)  : 0/0
    Level-2 CSNPs (sent/rcvd) : 1/1
    Level-2 PSNPs (sent/rcvd) : 0/0
    Level-2 LSP Flooding Duplicates     : 22919
    Level-2 LSPs Arrival Time Throttled : 0
  Interface TenGigE0/0/0/1.20:
    PTP Hellos (sent/rcvd)    : 760370/760306
    LSP Retransmissions       : 0
    LSP Peak Rate (out/in)    : 7/0
    Level-2 LSPs (sent/rcvd)  : 8/0
    Level-2 CSNPs (sent/rcvd) : 1/1
    Level-2 PSNPs (sent/rcvd) : 0/1
    Level-2 LSP Flooding Duplicates     : 22918
    Level-2 LSPs Arrival Time Throttled : 0
  Interface TenGigE0/0/0/2.10:
    PTP Hellos (sent/rcvd)    : 760223/760096
    LSP Retransmissions       : 4
    LSP Peak Rate (out/in)    : 6/2
    Level-2 LSPs (sent/rcvd)  : 42506/52552
    Level-2 CSNPs (sent/rcvd) : 4/4
    Level-2 PSNPs (sent/rcvd) : 52362/42363
    Level-2 LSP Flooding Duplicates     : 63
    Level-2 LSPs Arrival Time Throttled : 0
  Interface TenGigE0/0/0/2.20:
    PTP Hellos (sent/rcvd)    : 760363/760234
    LSP Retransmissions       : 2
    LSP Peak Rate (out/in)    : 6/0
    Level-2 LSPs (sent/rcvd)  : 13/7
    Level-2 CSNPs (sent/rcvd) : 4/4
    Level-2 PSNPs (sent/rcvd) : 6/1
    Level-2 LSP Flooding Duplicates     : 29708
    Level-2 LSPs Arrival Time Throttled : 0
  Interface TenGigE0/0/0/3.10:
    PTP Hellos (sent/rcvd)    : 760286/760231
    LSP Retransmissions       : 0
    LSP Peak Rate (out/in)    : 5/1
    Level-2 LSPs (sent/rcvd)  : 14/9
    Level-2 CSNPs (sent/rcvd) : 4/4
    Level-2 PSNPs (sent/rcvd) : 5/2
    Level-2 LSP Flooding Duplicates     : 29701
    Level-2 LSPs Arrival Time Throttled : 0
  Interface TenGigE0/0/0/3.20:
    PTP Hellos (sent/rcvd)    : 760121/760359
    LSP Retransmissions       : 9
    LSP Peak Rate (out/in)    : 1/0
    Level-2 LSPs (sent/rcvd)  : 22/6
    Level-2 CSNPs (sent/rcvd) : 4/4
    Level-2 PSNPs (sent/rcvd) : 5/1
    Level-2 LSP Flooding Duplicates     : 29696
    Level-2 LSPs Arrival Time Throttled : 0
  Interface TenGigE0/0/0/4.10:
    PTP Hellos (sent/rcvd)    : 414984/406274
    LSP Retransmissions       : 2
    LSP Peak Rate (out/in)    : 9/3
    Level-2 LSPs (sent/rcvd)  : 37830/10
    Level-2 CSNPs (sent/rcvd) : 5/5
    Level-2 PSNPs (sent/rcvd) : 0/37798
    Level-2 LSP Flooding Duplicates     : 3636
    Level-2 LSPs Arrival Time Throttled : 0
  Interface TenGigE0/0/0/4.20:
    PTP Hellos (sent/rcvd)    : 414927/406182
    LSP Retransmissions       : 0
    LSP Peak Rate (out/in)    : 8/4
    Level-2 LSPs (sent/rcvd)  : 11/40757
    Level-2 CSNPs (sent/rcvd) : 5/5
    Level-2 PSNPs (sent/rcvd) : 40495/3
    Level-2 LSP Flooding Duplicates     : 710
    Level-2 LSPs Arrival Time Throttled : 0

Traceback:

2024-04-29T13:18:08: %SCRIPT-ERROR: Traceback (most recent call last):
2024-04-29T13:18:08: %SCRIPT-ERROR:   File "/Users/chart2/Documents/Python/genieparser/src/genie/libs/parser/utils/unittests.py", line 580, in test_golden
2024-04-29T13:18:08: %SCRIPT-ERROR:     parsed_output = obj.parse(**arguments)
2024-04-29T13:18:08: %SCRIPT-ERROR:                     ^^^^^^^^^^^^^^^^^^^^^^
2024-04-29T13:18:08: %SCRIPT-ERROR:   File "src/genie/metaparser/_metaparser.py", line 342, in genie.metaparser._metaparser.MetaParser.parse
2024-04-29T13:18:08: %SCRIPT-ERROR:   File "src/genie/metaparser/_metaparser.py", line 322, in genie.metaparser._metaparser.MetaParser.parse
2024-04-29T13:18:08: %SCRIPT-ERROR:   File "src/genie/metaparser/util/schemaengine.py", line 419, in genie.metaparser.util.schemaengine.Schema.validate
2024-04-29T13:18:08: %SCRIPT-ERROR: genie.metaparser.util.exceptions.SchemaMissingKeyError: Missing keys: [['isis', '64512', 'transmit_time'], ['isis', '64512', 'process_time']]
SohanTirpude commented 2 weeks ago

Hello @ChristopherJHart,

It looks like a bug. Kindly give me some time to analyse this issue.

Thank you.

SohanTirpude commented 2 weeks ago

Hello @ChristopherJHart,

I have made required changes and now the parsed output is looking like this:

{
    "isis": {
        "64512": {
            "psnp_cache": {
                "hits": 91099,
                "tries": 146551
            },
            "csnp_cache": {
                "hits": 5,
                "tries": 30,
                "updates": 50
            },
            "lsp": {
                "checksum_errors_received": 0,
                "dropped": 0
            },
            "snp": {
                "dropped": 0
            },
            "upd": {
                "max_queue_size": 8,
                "queue_size": 0
            },
            "transmit_time": {
                "hello": {
                    "average_transmit_time_sec": 0,
                    "rate_per_sec": 2
                },
                "csnp": {
                    "average_transmit_time_sec": 0,
                    "rate_per_sec": 0
                },
                "psnp": {
                    "average_transmit_time_sec": 0,
                    "rate_per_sec": 0
                },
                "lsp": {
                    "average_transmit_time_sec": 0,
                    "rate_per_sec": 0
                }
            },
            "process_time": {
                "hello": {
                    "average_process_time_sec": 0,
                    "rate_per_sec": 2
                },
                "csnp": {
                    "average_process_time_sec": 0,
                    "rate_per_sec": 0
                },
                "psnp": {
                    "average_process_time_sec": 0,
                    "rate_per_sec": 0
                },
                "lsp": {
                    "average_process_time_sec": 0,
                    "rate_per_sec": 0
                }
            },
            "level": {
                2: {
                    "lsp": {
                        "new": 50,
                        "refresh": 16892
                    },
                    "address_family": {
                        "IPv4 Unicast": {
                            "total_spf_calculation": 7526,
                            "full_spf_calculation": 7508,
                            "ispf_calculation": 0,
                            "next_hop_calculation": 11,
                            "partial_route_calculation": 7,
                            "periodic_spf_calculation": 7382
                        }
                    }
                }
            },
            "interface": {
                "Loopback0": {},
                "TenGigE0/0/0/0.10": {
                    "level": {
                        2: {
                            "lsps_sourced": {
                                "sent": 49222,
                                "received": 53198,
                                "flooding_duplicates": 7,
                                "arrival_time_throttled": 0
                            },
                            "csnp": {
                                "sent": 1,
                                "received": 1
                            },
                            "psnp": {
                                "sent": 52991,
                                "received": 49049
                            }
                        }
                    }
                },
                "TenGigE0/0/0/0.20": {
                    "level": {
                        2: {
                            "lsps_sourced": {
                                "sent": 4,
                                "received": 12,
                                "flooding_duplicates": 22912,
                                "arrival_time_throttled": 0
                            },
                            "csnp": {
                                "sent": 1,
                                "received": 1
                            },
                            "psnp": {
                                "sent": 4,
                                "received": 3
                            }
                        }
                    }
                },
                "TenGigE0/0/0/1.10": {
                    "level": {
                        2: {
                            "lsps_sourced": {
                                "sent": 0,
                                "received": 0,
                                "flooding_duplicates": 22919,
                                "arrival_time_throttled": 0
                            },
                            "csnp": {
                                "sent": 1,
                                "received": 1
                            },
                            "psnp": {
                                "sent": 0,
                                "received": 0
                            }
                        }
                    }
                },
                "TenGigE0/0/0/1.20": {
                    "level": {
                        2: {
                            "lsps_sourced": {
                                "sent": 8,
                                "received": 0,
                                "flooding_duplicates": 22918,
                                "arrival_time_throttled": 0
                            },
                            "csnp": {
                                "sent": 1,
                                "received": 1
                            },
                            "psnp": {
                                "sent": 0,
                                "received": 1
                            }
                        }
                    }
                },
                "TenGigE0/0/0/2.10": {
                    "level": {
                        2: {
                            "lsps_sourced": {
                                "sent": 42506,
                                "received": 52552,
                                "flooding_duplicates": 63,
                                "arrival_time_throttled": 0
                            },
                            "csnp": {
                                "sent": 4,
                                "received": 4
                            },
                            "psnp": {
                                "sent": 52362,
                                "received": 42363
                            }
                        }
                    }
                },
                "TenGigE0/0/0/2.20": {
                    "level": {
                        2: {
                            "lsps_sourced": {
                                "sent": 13,
                                "received": 7,
                                "flooding_duplicates": 29708,
                                "arrival_time_throttled": 0
                            },
                            "csnp": {
                                "sent": 4,
                                "received": 4
                            },
                            "psnp": {
                                "sent": 6,
                                "received": 1
                            }
                        }
                    }
                },
                "TenGigE0/0/0/3.10": {
                    "level": {
                        2: {
                            "lsps_sourced": {
                                "sent": 14,
                                "received": 9,
                                "flooding_duplicates": 29701,
                                "arrival_time_throttled": 0
                            },
                            "csnp": {
                                "sent": 4,
                                "received": 4
                            },
                            "psnp": {
                                "sent": 5,
                                "received": 2
                            }
                        }
                    }
                },
                "TenGigE0/0/0/3.20": {
                    "level": {
                        2: {
                            "lsps_sourced": {
                                "sent": 22,
                                "received": 6,
                                "flooding_duplicates": 29696,
                                "arrival_time_throttled": 0
                            },
                            "csnp": {
                                "sent": 4,
                                "received": 4
                            },
                            "psnp": {
                                "sent": 5,
                                "received": 1
                            }
                        }
                    }
                },
                "TenGigE0/0/0/4.10": {
                    "level": {
                        2: {
                            "lsps_sourced": {
                                "sent": 37830,
                                "received": 10,
                                "flooding_duplicates": 3636,
                                "arrival_time_throttled": 0
                            },
                            "csnp": {
                                "sent": 5,
                                "received": 5
                            },
                            "psnp": {
                                "sent": 0,
                                "received": 37798
                            }
                        }
                    }
                },
                "TenGigE0/0/0/4.20": {
                    "level": {
                        2: {
                            "lsps_sourced": {
                                "sent": 11,
                                "received": 40757,
                                "flooding_duplicates": 710,
                                "arrival_time_throttled": 0
                            },
                            "csnp": {
                                "sent": 5,
                                "received": 5
                            },
                            "psnp": {
                                "sent": 40495,
                                "received": 3
                            }
                        }
                    }
                }
            }
        }
    }
}

Kindly check and let me know if something is missing.

Thank you.

ChristopherJHart commented 1 week ago

@SohanTirpude This looks great - thank you!

SohanTirpude commented 3 days ago

Hello @ChristopherJHart,

The PR got merged now and the fix will be available in the upcoming pyats 24.5 release. Hence, I am closing this ticket for now. Once pyats 24.5 version gets released then kindly try this and let me know if it is not working as it should. If you are facing any other issues, kindly open a new ticket.

Thank you.