Sorting is evaluating strings with \u0000 Unicode-escaped sequences

[ZStoner]

I tried to do a sort on some JSON that contains strings with Unicode-escaped sequences. Note: The Unicode-escaped strings weren't part of any "Keys" used for the sort criteria.

Issues:

• The resulting sort is physically altering the JSON data by replacing \u0000 escape sequences with their evaluated Unicode Character.
• Reference: Sample JSON data used below was taken from: GitHub Flavored Markdown Spec: Entity References
• I tested with some other "Low ASCII" sequences (...below char 32), and those that don't already have a pre-defined escape (...like \t,\r,\n), may also get evaluated into an undesired character.
  • In my case, I tested with "\u0017", and sort converted it to a control symbol that fails JSON parsing.

Sample JSON (before Sort):

{
  "&AElig": { "codepoints": [198], "characters": "\u00C6" },
  "Æ": { "codepoints": [198], "characters": "\u00C6" },
  "↓": { "codepoints": [8595], "characters": "\u2193" },
  "⤓": { "codepoints": [10515], "characters": "\u2913" },
  "⊤": { "codepoints": [8868], "characters": "\u22A4" },
  "↧": { "codepoints": [8615], "characters": "\u21A7" },
  "⇓": { "codepoints": [8659], "characters": "\u21D3" }
}

Resulting JSON (after Sort Ascending):

{
  "&AElig": {
    "codepoints": [
      198
    ],
    "characters": "Æ"
  },
  "Æ": {
    "codepoints": [
      198
    ],
    "characters": "Æ"
  },
  "↓": {
    "codepoints": [
      8595
    ],
    "characters": "↓"
  },
  "⇓": {
    "codepoints": [
      8659
    ],
    "characters": "⇓"
  },
  "⤓": {
    "codepoints": [
      10515
    ],
    "characters": "⤓"
  },
  "⊤": {
    "codepoints": [
      8868
    ],
    "characters": "⊤"
  },
  "↧": {
    "codepoints": [
      8615
    ],
    "characters": "↧"
  }
}

[R35007]

@ZStoner,

Thanks for pointing out this issue. I will try to resolve it in release and will update you :)

[R35007]

Hi @ZStoner ,

Since the selected text to sort is converted to JavaScript object all the Unicode will be converted to it's character by default. To overcome this, I have given a fix which may not work for all the Unicode's.

I have added a new vscode sort setting sort-json.settings.preserveUnicodeString. This helps to preserve the Unicode string. Please use this with caution.

Please update the Sort JSON to version 15 Closing this ticket.

Thanks, Siva