unexpected token

[AngryBerryMS]

When I tried this extension with the following Json file, it encountered a unexpected token error.

Based on this answer, could you please handle when the case have some hidden/special characters?

export default function parseArray(array: string, fileExtension: FileExtension) : unknown[] {
  let object;
  try {
    object = JSON.parse(applyPreFilter(array.replace(/\\n/g, "\\n")  
               .replace(/\\'/g, "\\'")
               .replace(/\\"/g, '\\"')
               .replace(/\\&/g, "\\&")
               .replace(/\\r/g, "\\r")
               .replace(/\\t/g, "\\t")
               .replace(/\\b/g, "\\b")
               .replace(/\\f/g, "\\f");, fileExtension));
  } catch (e) {
    throw new Error(`Cannot parse selection as JSON. Reason: ${e}`);
  }
  if (object.constructor === Array) {
    return object;
  } else {
    throw new Error(`Selection is a ${object.constructor} not an array.`);
  }
}

JSON file for test:

{
    "redirections": [
      {
        "source_path": "a/about.md",
        "redirect_url": "/test/about",
        "redirect_document_id": false
      },
      {
        "source_path": "b/javascript.md",
        "redirect_url": "/test/js",
        "redirect_document_id": false
      },
      {
        "source_path": "b/javascript.md",
        "redirect_url": "/test/aaa",
        "redirect_document_id": false
      }
  ]
}

[fvclaus]

Thanks for creating an issue and providing a fix as well!

I am having problems understanding the issue. I read through the linked StackOverflow answer as well as all comments, but still don't understand it. There seems to be a problem with special characters, but the code does not actually replace special characters. Using \\n as an example:

• \\n is a literal \ followed by an n, therefore printed as \n
• The node repl will "render" it as \\n, unless it is printed to console.log

  > '\\n'
  '\\n'
  > console.log('\\n')
  \n
  > "\\n".replace(/\\n/, "\\n")
  '\\n'
  > console.log("\\n".replace(/\\n/, "\\n"))
  \n

  So the code is replacing a \\n with a \\n. That does not change anything.

Looking at your example, there seems to be a problem with whitespace, but I can't figure it out. Pasting your array to an vsc or any other editor shows me normal whitespaces and the array can be parsed successfully.

Can you upload your example file that is not working?

[AngryBerryMS]

Sorry for the late reply.

I found an easy way to resolve this probelm, just press ctrl+shift+p in vs code and enter format, and after formatting this extension can work properly.

[fvclaus]

Sure. That is a workaround. Do you still have the file in a state where the parsing fails? I would like to take a look at it. Others might have the same issue and I was planning to write a more forgiving parser anyway.

[AngryBerryMS]

Yes. You can use this file to reproduce the problem (download and remove the .txt extension)

openpublishing.redirection.test.json.txt

The vertical lines used to show the hierarchy are broken in the red box part.

[fvclaus]

Thanks for sharing the file. The white space before the last array element are not normal spaces but a non-breaking spaces. See: https://en.wikipedia.org/wiki/Non-breaking_space

Non-breaking spaces are not valid white spaces in JSON (or any language I know). I will add code that can handle them, because that might be a common issue. I will leave the issue open until this is fixed as a reminder for me and for other people to see.

[fvclaus]

This is a reminder for myself: JS supports even more whitespace characters. A list can be found here: https://tc39.es/ecma262/#sec-white-space Below are the line terminators that are supported. I should support them as well.