highlighting is incorrect with literal `#\"`

dannypsnl commented 3 years ago

Environment

OS: MacOS
OS Version BigSur
VS Code Version 1.58.2
Racket version 8.2
- run: racket -v
Racket Langserver checksum 6d01f910dfd332d3bcaeef176162de80931b324a
- run: raco pkg show --full-checksum racket-langserver
Magic Racket version 0.5.7

Additional Context

jryans commented 2 years ago

A related example from Scribble which seems to hit the same issue:

(define (write-string/xml-quote str p [start 0] [end (string-length str)])
  (let loop ([start start])
    (when (< start end)
      (define m (regexp-match-positions #rx"[&<>\"]" str start end p))
      (when m
        (write-string (case (string-ref str (caar m))
                        [(#\&) "&amp;"]
                        [(#\<) "&lt;"]
                        [(#\>) "&gt;"]
                        [(#\") "&quot;"])
                      p)
        (loop (cdar m))))))

dannypsnl commented 2 years ago

@Eugleo would you mind giving some hints?

Eugleo commented 2 years ago

Sure. In general, it might be useful to look at the Developer: Inspect Editor Tokens and Scopes command. If you're not sure what a token and a scope is, I recommend giving this article in VSC docs a look.

I found out that the character literals actually work ok when used at top-level, and the causes are different in each case that you presented.

First problem

The problem in @jryans code is the #rx string in which the first \" isn't properly escaped. To fix this, we can add the following pattern to the regexp-string pattern:

"patterns": [
  {
    "include": "#escape-char-base"
  }
]

The same piece of code is in byte strings and normal strings. I'm not sure why I haven't added support for escape characters into regex strings in the first place — maybe I just forgot to do so. Anyway, once I add this, the highlighting in this case is fixed:

Second problem

As for the original problem, I have no idea what's causing it. On my machine, it works fine:

I'm also on MacOS, Racket 8.3, Magic Racket 5.7.1. Please make sure to update your magic racket to the latest version, and also make sure you only have magic racket a no other racket extension installed. If the problem persists, please send me the whole file and also tell me what does the token and scope command show for that problematic piece of code.

Eugleo commented 2 years ago

In the meantime, I will close this issue because I'm unable to reproduce it.

dannypsnl commented 2 years ago

My problem get fixed with Magic-Racket 5.8, thanks

Eugleo / magic-racket