ZuxingGit / pygments

Pygments is a generic syntax highlighter written in Python
http://pygments.org/
BSD 2-Clause "Simplified" License
0 stars 0 forks source link

#2686 #8

Open trungtruong1 opened 4 months ago

trungtruong1 commented 4 months ago

I found another issue in the repo, which is #2686. It is a relatively new issue.

trungtruong1 commented 4 months ago

I managed to fix this issue.

Before: image

After: image

I created another branch and pushed the changes. This issue is about TCL and I have some knowledge about this.

IshaanVerma1130 commented 4 months ago

i'll run tests for your commit.

trungtruong1 commented 4 months ago

Thanks! @IshaanVerma1130

IshaanVerma1130 commented 4 months ago

image It is still marking it as TEXT when it should be TEXT>Whitespace @trungtruong1

trungtruong1 commented 4 months ago

I think \ and $ shouldn't be marked as TEXT.Whitespace

IshaanVerma1130 commented 4 months ago

It's '\n' in the input code. But it's reading it separately as '\' and 'n'.

I believe it should be reading them as newlines as indicated by the comments in the code.

trungtruong1 commented 4 months ago

Ah I see, missed that part, I'll fix it

trungtruong1 commented 4 months ago

Hello, I made a simple fix to the issue. This is the new output for a test file: ExampleRun

Also, I think it is better to mark \(.) as String.Escape. I use a highlighter in VSCode for Tcl and found out that they do it too. Highlight by VSCode Extension This is the code highlighted by the extension. They counted the brackets } incorrectly but that does not apply to our case.

This is my result after the fix: NewHighlighted

I believe this fixed a HACK in the lexer as well. The fix is in 0285053

IshaanVerma1130 commented 4 months ago

I believe it should still be marked as TEXT.WHITESPACE. Looking at the other Lexers and what they do. Also according to this message, the issue says that some characters should be read as escape character: image and just below that it said the sometimes it adds special meanings probably telling that it should be read as newline or TEXT.WHITESPACE according to the Pygments methodology. image

Take a look at this: https://www.learnxinyminutes.com/docs/tcl/ Want to clarify this with the author?

trungtruong1 commented 4 months ago

I think asking them would be nice, would you mind adding a comment to the issue asking for clarification?

IshaanVerma1130 commented 4 months ago

The author still hasn't replied. T_T

IshaanVerma1130 commented 3 months ago

image image @trungtruong1 The \n at the last should be recognized as TEXT.WHITESPACE but is instead marked as TEXT. Please check. The other \\n are correctly marked.

trungtruong1 commented 3 months ago

That last \n is lexed using the original logic of the lexer. My modification did not result in that. You can check the output example in this: image

https://github.com/ZuxingGit/pygments/blob/2686/tests/snippets/tcl/test_vars.txt