search

jgm / skylighting

A Haskell syntax highlighting library with tokenizers derived from KDE syntax highlighting descriptions
189 stars 61 forks source link

ini file highlighting doesn't preserve highlight for strings #126

Closed jjallaire closed 3 years ago

jjallaire commented 3 years ago

With pandoc 2.14.0.2 if you render the following markdown:

```ini
[Section]
Field = "something-v4.2.4.yaml"


You get this highlighting treatment:

<img width="346" alt="image (6)" src="https://user-images.githubusercontent.com/104391/122104154-5dcf3e80-cde5-11eb-9eef-36e250a4011e.png">

Note the "2.4" highlighted as a float rather than preserving the string highlighting color.
jgm commented 3 years ago

I'll move this to skylighting (which does the highlighting). We can see what's going on this way:

 % skylighting --syntax ini --trace
[Section]
Field = "something-v4.2.4.yaml"
Trying rule Rule {rMatcher = RangeDetect '[' ']', rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
RangeDetect MATCHED Just (KeywordTok,"[Section]")
Trying rule Rule {rMatcher = RangeDetect '[' ']', rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = DetectChar '=', rAttribute = OtherTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = [Push ("INI Files","Value")]}
Trying rule Rule {rMatcher = AnyChar (fromList "#;"), rAttribute = CommentTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = True, rColumn = Nothing, rContextSwitch = [Push ("INI Files","Comment")]}
FALLTHROUGH Just (DataTypeTok,"Field")
Trying rule Rule {rMatcher = RangeDetect '[' ']', rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = DetectChar '=', rAttribute = OtherTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = [Push ("INI Files","Value")]}
Trying rule Rule {rMatcher = AnyChar (fromList "#;"), rAttribute = CommentTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = True, rColumn = Nothing, rContextSwitch = [Push ("INI Files","Comment")]}
FALLTHROUGH Just (DataTypeTok," ")
Trying rule Rule {rMatcher = RangeDetect '[' ']', rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = DetectChar '=', rAttribute = OtherTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = [Push ("INI Files","Value")]}
DetectChar MATCHED Just (OtherTok,"=")
CONTEXT STACK ["Value","ini"]
Trying rule Rule {rMatcher = Float, rAttribute = FloatTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Int, rAttribute = DecValTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Keyword (KeywordAttr {keywordCaseSensitive = False, keywordDelims = fromList "\t\n !%&()*+,-./:;<=>?[\\]^{|}~"}) (CaseInsensitiveWords (fromList ["default","defaults","e_all","e_compile_error","e_compile_warning","e_core_error","e_core_warning","e_error","e_notice","e_parse","e_strict","e_user_error","e_user_notice","e_user_warning","e_warning","false","localhost","no","normal","null","off","on","true","yes"])), rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
FALLTHROUGH Just (StringTok," ")
Trying rule Rule {rMatcher = Float, rAttribute = FloatTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Int, rAttribute = DecValTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Keyword (KeywordAttr {keywordCaseSensitive = False, keywordDelims = fromList "\t\n !%&()*+,-./:;<=>?[\\]^{|}~"}) (CaseInsensitiveWords (fromList ["default","defaults","e_all","e_compile_error","e_compile_warning","e_core_error","e_core_warning","e_error","e_notice","e_parse","e_strict","e_user_error","e_user_notice","e_user_warning","e_warning","false","localhost","no","normal","null","off","on","true","yes"])), rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
FALLTHROUGH Just (StringTok,"\"")
Trying rule Rule {rMatcher = Float, rAttribute = FloatTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Int, rAttribute = DecValTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Keyword (KeywordAttr {keywordCaseSensitive = False, keywordDelims = fromList "\t\n !%&()*+,-./:;<=>?[\\]^{|}~"}) (CaseInsensitiveWords (fromList ["default","defaults","e_all","e_compile_error","e_compile_warning","e_core_error","e_core_warning","e_error","e_notice","e_parse","e_strict","e_user_error","e_user_notice","e_user_warning","e_warning","false","localhost","no","normal","null","off","on","true","yes"])), rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
FALLTHROUGH Just (StringTok,"something")
Trying rule Rule {rMatcher = Float, rAttribute = FloatTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Int, rAttribute = DecValTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Keyword (KeywordAttr {keywordCaseSensitive = False, keywordDelims = fromList "\t\n !%&()*+,-./:;<=>?[\\]^{|}~"}) (CaseInsensitiveWords (fromList ["default","defaults","e_all","e_compile_error","e_compile_warning","e_core_error","e_core_warning","e_error","e_notice","e_parse","e_strict","e_user_error","e_user_notice","e_user_warning","e_warning","false","localhost","no","normal","null","off","on","true","yes"])), rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
FALLTHROUGH Just (StringTok,"-")
Trying rule Rule {rMatcher = Float, rAttribute = FloatTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Int, rAttribute = DecValTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Keyword (KeywordAttr {keywordCaseSensitive = False, keywordDelims = fromList "\t\n !%&()*+,-./:;<=>?[\\]^{|}~"}) (CaseInsensitiveWords (fromList ["default","defaults","e_all","e_compile_error","e_compile_warning","e_core_error","e_core_warning","e_error","e_notice","e_parse","e_strict","e_user_error","e_user_notice","e_user_warning","e_warning","false","localhost","no","normal","null","off","on","true","yes"])), rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
FALLTHROUGH Just (StringTok,"v4")
Trying rule Rule {rMatcher = Float, rAttribute = FloatTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Int, rAttribute = DecValTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Keyword (KeywordAttr {keywordCaseSensitive = False, keywordDelims = fromList "\t\n !%&()*+,-./:;<=>?[\\]^{|}~"}) (CaseInsensitiveWords (fromList ["default","defaults","e_all","e_compile_error","e_compile_warning","e_core_error","e_core_warning","e_error","e_notice","e_parse","e_strict","e_user_error","e_user_notice","e_user_warning","e_warning","false","localhost","no","normal","null","off","on","true","yes"])), rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
FALLTHROUGH Just (StringTok,".")
Trying rule Rule {rMatcher = Float, rAttribute = FloatTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Int, rAttribute = DecValTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Int MATCHED Just (DecValTok,"2")
Trying rule Rule {rMatcher = Float, rAttribute = FloatTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Int, rAttribute = DecValTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Keyword (KeywordAttr {keywordCaseSensitive = False, keywordDelims = fromList "\t\n !%&()*+,-./:;<=>?[\\]^{|}~"}) (CaseInsensitiveWords (fromList ["default","defaults","e_all","e_compile_error","e_compile_warning","e_core_error","e_core_warning","e_error","e_notice","e_parse","e_strict","e_user_error","e_user_notice","e_user_warning","e_warning","false","localhost","no","normal","null","off","on","true","yes"])), rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
FALLTHROUGH Just (StringTok,".")
Trying rule Rule {rMatcher = Float, rAttribute = FloatTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Float MATCHED Just (FloatTok,"4.")
Trying rule Rule {rMatcher = Float, rAttribute = FloatTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Int, rAttribute = DecValTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Keyword (KeywordAttr {keywordCaseSensitive = False, keywordDelims = fromList "\t\n !%&()*+,-./:;<=>?[\\]^{|}~"}) (CaseInsensitiveWords (fromList ["default","defaults","e_all","e_compile_error","e_compile_warning","e_core_error","e_core_warning","e_error","e_notice","e_parse","e_strict","e_user_error","e_user_notice","e_user_warning","e_warning","false","localhost","no","normal","null","off","on","true","yes"])), rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
FALLTHROUGH Just (StringTok,"yaml")
Trying rule Rule {rMatcher = Float, rAttribute = FloatTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Int, rAttribute = DecValTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
Trying rule Rule {rMatcher = Keyword (KeywordAttr {keywordCaseSensitive = False, keywordDelims = fromList "\t\n !%&()*+,-./:;<=>?[\\]^{|}~"}) (CaseInsensitiveWords (fromList ["default","defaults","e_all","e_compile_error","e_compile_warning","e_core_error","e_core_warning","e_error","e_notice","e_parse","e_strict","e_user_error","e_user_notice","e_user_warning","e_warning","false","localhost","no","normal","null","off","on","true","yes"])), rAttribute = KeywordTok, rIncludeAttribute = False, rDynamic = False, rCaseSensitive = True, rChildren = [], rLookahead = False, rFirstNonspace = False, rColumn = Nothing, rContextSwitch = []}
FALLTHROUGH Just (StringTok,"\"")
checkLineEnd for "Value" eol = True cLineEndContext = [Pop]
CONTEXT STACK ["ini"]
jgm commented 3 years ago

The problem is in the KDE ini.xml syntax definition, which seems to be very primitive. I tried in the Kate editor and got the same bad highlighting:

Screen Shot 2021-06-15 at 10 50 07 PM

So, we are implementing this correctly but the underlying syntax definition needs improvement. (If you improve it, submit upstream to KDE as described in this project's README.)

jjallaire commented 3 years ago

Okay, we will take a shot at improving this and submit to KDE once we have. Thanks!

jjallaire commented 3 years ago

Issue w/ proposed fix filed here: https://invent.kde.org/frameworks/syntax-highlighting/-/issues/10

jgm commented 3 years ago

Great. Let's see if you get any reactions from KDE. If they're okay with it, we can merge it here too.

jgm commented 3 years ago

Thinking more about this, it seems to me that instead of adding special support for quoted strings, you should remove the special highlighting of numbers. Reasons:

  1. The description of ini file format here https://en.wikipedia.org/wiki/INI_file#Quoted_values says nothing at all about special treatment of numbers.
  2. With the fix you've given, we'd still get similar problems with numbers in the middle of unquoted strings, which are allowed in ini.
jjallaire commented 3 years ago

Okay, agreed. I re-opened the original issue w/ a new suggested patch: https://invent.kde.org/frameworks/syntax-highlighting/-/issues/10#note_262011

jjallaire commented 3 years ago

It looks like they've merged an improved version that only colorizes numbers for values that are entirely composed of a number: https://invent.kde.org/frameworks/syntax-highlighting/-/merge_requests/224

jgm commented 3 years ago

Great, I've merged this now.