Open joewesch opened 2 years ago
So Genie tries to guess based on weighting what the abbreviated command represents?
Should we just enable this by default? Are there any downsides to do doing this?
I am not seeing much downside in just enabling this.
I took every command in their parsers.json file and ran them through the ._fuzzy_search_command()
with both fuzzy set to False
(the current setting) and then True
(the proposed change) and here are my findings:
=======================================================
With regards to using full syntax commands (where there are no variables), I was unable to detect a change. Every parser result returned matched exactly the command that I sent in. This is to be expected as the exact syntax matching is weighed higher than partial syntax or variables.
=======================================================
With regards to commands with variables in them, I substituted every variable with the word "variable" (via re.sub(r"{.*?}", "variable", command)
). There were 74 false positives when fuzzy was set to False
.
For example:
Command: ping variable variable
Expected result: ping {addr} {intf_name}
Result: ping {addr} {count}
Command: show bgp vrf variable variable
Expected result: show bgp vrf {vrf} {route}
Result: show bgp vrf {vrf} {address_family}
Command: show idprom interface variable
Expected result: show idprom interface {mode}
Result: show idprom interface {interface}
Command: show interface variable counters
Expected result: show interface {intf} counters
Result: show interface {interface} counters
# Etc.
However, when changing fuzzy to True
it produced the exact same 74 false positives. My rudimentary substitution of the word "variable" is probably not the best way to test this, but it did prove that changing fuzzy to True
produced the same false positives as False
.
However, was when setting fuzzy to True
the following false negatives did occur whereas they had met expectations when fuzzy was set to False
:
Command: show log variable | except variable | match variable
Expected result: show log {filename} | except {except_} | match {match}
No Result
Command: show log variable | match variable | except variable
Expected result: show log {filename} | match {match} | except {except_}
No Result
Command: show running-config variable | sec '^i' | inc variable
Expected result: show running-config {pim} | sec '^i' | inc {pip_str}
No Result
Command: show running-config variable | sec variable | inc variable
Expected result: show running-config {pim} | sec {vrf} | inc {pip_str}
No Result
Again, this could be because of the way I am brute force testing, so not 100% sure.
=======================================================
Lastly, I took every command that didn't have a variable in it again and removed the last letter of each word in the command to simulate partial syntax commands at every word. When I ran them through with fuzzy=False
it produced 261 failures (244 ambiguous exceptions and 17 false positives).
When I ran it with fuzzy=True
it produced 62 failures (61 false positives and 1 false negative). Quite a bit less than previous.
So, we have false positives...but when looking over the false positives all of them were because that last letter of the syntax is indeed needed. For example:
Full command: show bfd sessions
Partial command: sho bf session
Result: show bfd session
Full command: show bgp neighbors
Partial command: sho bg neighbor
Result: show bgp neighbor
Full command: show interfaces summary
Partial command: sho interface summar
Result: show interface summary
And lastly, as I stated earlier it did produce a single false negative:
Full command: show platform software process slot switch active R0 monitor | inc Mem :|Swap:
Partial command: sho platfor softwar proces slo switc activ R0 monito | in Me :|Swap:
No Result
=======================================================
So, in conclusion: setting fuzzy to True
by default seems to correct well over 200 ambiguous errors for partial syntax, but may cause a few false negatives by doing so.
When using the
.send_command()
method withuse_genie=True
the command syntax being sent can sometimes be abbreviated (i.e.sh ip int
vsshow ip interface
), but not always (sh ip int br
fails). After digging through the Genie code I was able to determine that it was due to the way they score/weigh each word in the command. I initially created an issue on their repo thinking this was a bug, but realized afterward that this was because I was calling their._fuzzy_search_command()
function withfuzzy=False
as this is the default and there is no way in Netmiko to enable this option.It would be nice to have an option on the
.send_command()
method that can be passed to Genie to enable fuzzy syntax matching.