[Feature request] Capability to display non-ascii characters?

[mark9804]

Current Syntax Highlight Version

2.1.10

macOS Version

Monterey 12.2.1 (21D62)

Feature Description

Hi sbarex,

When I try to preview a JSON file with non-ascii (in my case, CJK characters) characters present, they will be parsed into ascii-safe form (e.g. \u6d4b\u8bd5…). If possible, would you kindly implement an option to allow SyntaxHighlight to display non-ascii characters?

Screenshot

Test case

{
    "en": "test",
    "ch": "测试",
    "ja": "テスト",
    "ko": "테스트"
}

[sbarex]

On my system (intel macOS 12.3) that json code, saved as utf-8, is displayed correctly.

Can you send me your file to see what encoding it uses?

[mark9804]

I created with and saved the file as utf-8 with vscode too on my intel macOS 12.3 (21E230). The file link goes as follows.

https://www.dropbox.com/s/spzqi41pfgj41fl/SyntaxHighlight%20Non-ascii%20encoding%20test.json?dl=0

I removed all custom generators (qlImageSize, QLMarkdown, QLStephen) and refreshed quicklook with qlmanage -r and restarted system to make sure content rendering is really controlled by SyntaxHighlight. I confirmed this by changing SyntaxHighlight's theme and saw the theme had changed.

[sbarex]

Which rendering engine do you use: html or rtf? On macOS 12 I recommended using the html engine.

The bug is only for JSON files? if true, in Settings > Formats > JSON, remove the preprocessor setting if it is set.

If this not resolve, in Settings enable the debug option. The open a quick look preview, then please send me the colorize.html file generated on your desktop.

[mark9804]

1. I'm using HTML render engine.
2. I tried with (plain text file, javascript and python) and this doesn't happen. I guess it's a JSON only bug. No special settings for JSON is set.
   JSON format setting screenshot

3. I will upload a bundled file including:

• colorize.html
• colorize.log
• original JSON

colorize-syntax-highlight-nonascii.zip

[sbarex]

ok, mark the preprocessor option and put a single space as the value.

[mark9804]

ok, mark the preprocessor option and put a single space as the value.

This step produces the correct output. Now the content is shown in CJK characters.

[sbarex]

The preprocessor was used to beautify for json files. In the next release I will remove this default option that cause a lot of problems.

[mark9804]

Thanks for the work!

[sbarex]

Fixed on release 2.1.11.