Highlighter Example

[JesterOrNot]

After a few hours of reading the documentation and looking around online I was unable to understand how to implement a highlighter with rustyline can you give me a minimal example?

[JesterOrNot]

Is there an update on this?

[gwenn]

There is a basic example here to highlight matching brackets. You can also search all dependent crates for more advanced examples: https://github.com/search?q=language%3Arust+rustyline+impl+Highlighter+syntax&type=Code Like https://github.com/nushell/nushell/blob/f8be1becf28e29b728bd7cf481d3c717b0831751/src/shell/helper.rs#L67

And you should be aware that, currently, highlighting is not always reevaluated each time you press a key at the end of line. Furthermore, we would like to introduce a Tokenizer to avoid reparsing whole input each time we do completion / highlighting / hinting / validation. So the API may change...

[JesterOrNot]

I was thinking more of an example of how to make your own like if I want to make it so that the exit is yellow or something like that. Instead of just showing the built-in ones

[JesterOrNot]

See https://github.com/JesterOrNot/rustsh

[gwenn]

What's wrong with: https://github.com/nushell/nushell/blob/f8be1becf28e29b728bd7cf481d3c717b0831751/src/shell/helper.rs#L67 ?

[JesterOrNot]

Nothing, from time to time I help with nushell, I think it's awesome, but it's hard to replicate they use external libraries for parsing, etc I am proposing that you show a minimal example of how to create your own syntax highlighter with rustyline for people to build off of.

[JesterOrNot]

See https://github.com/nushell/nushell/tree/master/crates/nu-parser

[gwenn]

I would prefer to add such minimal example when we have a Tokenizer.

[JesterOrNot]

Can I contribute to the Tokenizer I really like this library

[gwenn]

Sure ! There is no draft but only comments here:

• https://github.com/kkawakam/rustyline/issues/207#issuecomment-585351357
• https://github.com/kkawakam/rustyline/issues/197#issuecomment-458282974

[JesterOrNot]

What still needs to be done? Can we create a draft PR?

[gwenn]

A draft PR would be perfect.

[JesterOrNot]

Is there a branch where it is being worked on?

[gwenn]

Nope.

[JesterOrNot]

So what is the idea behind the Tokenizer so that I can get started

[gwenn]

Provide a TokenTree / Document instead of a simple String for the Completer / Highlighter / Hinter / Validator.

[JesterOrNot]

So basically a lexer?

[gwenn]

An API compatible with an incremental lexer / parser. We should pass the whole line in case the lexer / parser is not incremental and a change. We should then have access to an AST / Document which can be used by Completer / Highlighter / Hinter / Validator.