`lexers.DelegatingLexer` always does not highlight tokens correctly

Delta456 commented 5 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug

I am currently working on a feature for charmbracelet/freeze which makes use of the chroma library and I am using the DelegatingLexer where I am passing two languages py and go as the language and root respectively. My PR implementation can be seen here: https://github.com/charmbracelet/freeze/pull/79

Using

./freeze --language py,go file.py gives
./freeze --language go,py file.py gives

As it can be seen chroma doesn't highlight the root of the lexers.DelegatingLexer. Only the language part is highlighted. However, this is not always the case.

./freeze --language jinja,html file.jinja gives

./freeze --language jinja file.jinja gives

In this case, jinja,html languages correctly highlight the code snippet.

To Reproduce

Chroma Playground doesn't support syntax highlighting of two languages at one time.

To locally reproduce this:

Clone https://github.com/Delta456/freeze/tree/multi_lang
Run go build .
Run ./freeze --language py,go file.py

Contents of file.py:

def hello():
    print('hello world')

func Hello() {
    fmt.Println("Hello world")
}

isabelroses commented 5 months ago

Also noticing the issue, particularly with PHP and html, with any any combination of PHP or html being the root.

alecthomas commented 5 months ago

@Delta456 the delegating lexer is working as intended. If you read the documentation it describes how it works, in that the root lexer must return a token type of Other. Neither Go or Python are designed for this, and I'm not sure how that would even work.

alecthomas commented 5 months ago

@isabelroses if you find a bug please file an issue. There are tests for the delegating lexer, and the "phtml" lexer uses it directly and nobody has filed any bugs for a few years, but anything's possible.

Delta456 commented 5 months ago

@Delta456 the delegating lexer is working as intended. If you read the documentation it describes how it works, in that the root lexer must return a token type of Other. Neither Go or Python are designed for this, and I'm not sure how that would even work.

That's true indeed. Can you add an explicit list of supported explicit languages for DelegatingLexer?

alecthomas commented 5 months ago

No, but if you wish to send a PR with an update to the README I'd consider it.

alecthomas / chroma