Gauging interest for visual debug feature

[marrable]

Hi, I really like this project (I use it at work parsing responses from routers/switches). I was wondering if anyone would be happy for me to add a visual debug feature? It would work by taking the typical two input arguments (template and cli text) plus a flag like '--visual-debug' and spit out the cli text with coloured highlighting, indicating matches and state transitions. I think it would be useful as a debug feature to show any errors in the template implementation. It would either produce an html doc of the cli text using css for the highlighting or just push the coloured cli text to stdout, whichever you prefer. Thanks!

[buxtronix]

Sounds interesting, I'd be keen to see a proposal for this.

[marrable]

@buxtronix great! Here is a rough proposal in the form of a concept HTML file. I would add a class to keep a record of the parsing procedure, capturing state transitions, regex matches and where they occurred in the CLI text. This would then be the input to a basic HTML builder that would spit out a single highlighted file with embedded CSS to keep things simple. Mousing over regex matches would show the regex along with the value that it is linked to. I wouldn't be using any third party libs and would keep the HTML builder as simple as possible.

[marrable]

Here's an output html from a short prototype run on a large template/cli text. I realise that debugging in code is just as viable for most applications, but I think this would help when authoring complex templates for large outputs. https://drive.google.com/file/d/1M2qZQkpqdwD9xGLxltPzVtKoGoDyev8w/view?usp=sharing

If you don't think the added complexity would help I completely understand 👍

[marrable]

To demonstrate its value as a tool, I had a situation this morning where we were getting errors in production related to CLI text parsing. The issue was a buggy template where the rule in one of the states only accounted for IPv4 and not v6. This is really clear when you look at the debug output:

Typically the "brown state" would last for one or two lines, so it stuck out like a sore thumb with the colour coded state transitions.

[buxtronix]

Useful! Please send a PR through and we'll take a look at it for potential integration here.

[marrable]

@buxtronix sure thing! I'll just need to pretty it up for submission. Should be in the next few days 👍

[0x5a17ed]

What happened to the debugging tool? This issue was closed while the PR is still open: #48

[harro]

Good question - perhaps @marrable can comment?

[marrable]

I closed it to move discussion to the PR, maybe a bit naive of me. Just didn't want to create too much noise over a feature that I wasn't sure had any interest. Happy to reopen and finish work on this if people think it's useful! @harro

[marrable]

When there was no response on the PR I assumed that it wasn't really a feature you thought was worth additional complexity or my implementation wasn't appropriate

[harro]

Looks interesting!

The aspect that I wish I'd caught when the proposal was first floated (on Jan 14) is that the mock suggested could have been rendered using terminal.py here in textfsm i.e. it could have been possible to render the colour formatted debug straight to the tty, avoiding the need for writing intermediary files and graphical support.

[harro]

But lets run with the HTML version you have ... some comments in the Pull request

[harro]

Ping?

[marrable]

Hey @harro, sorry for going dead on this, was busy finishing my degree! I am now free to work on it, and will aim to implement your suggestions in the next two weeks.

[harro]

Sounds good!

[marrable]

@harro so I have been going over my previous work and taking into account what you said about malicious HTML/JS. I have tried both lynx and w3m as potential in-terminal renderers, but there's a lot that gets lost and it seems like a sub-optimal solution.

I am considering doing a rewrite that just uses terminal colour printing instead of HTML. What are your thoughts on that? I know there can be issues with limited palettes on Windows, but it seems like the best route.

[harro]

Using terminal colors would seem like the best way to show feedback inline and accelerate the turnaround on debugging. Not sure how discernible the differences are for color blind users?

[cldeluna]

This feature would be incredibly useful!