Parser overhaul

[molsonkiko]

MAJOR CHANGE

Parsing is completely overhauled in version 5.0.0. Here are the key changes:

1. The parser tracks the cursor position of JSON elements instead of the line number.
   • This is the cursor position of the start of the element in the UTF-8 encoding of the document.
   • This makes navigating the document using the tree view much better because it is useful for navigation even when the entire document is one line.
2. Errors while parsing need not throw errors
   • Instead the parser has a state attribute that tracks how well the JSON string complies with JSON specifications of varying degrees of strictness.
   • The parser also has a logger_level attribute that determines how strictly the parser will enforce the JSON spec.
   • If the parser's state ever reaches FATAL, parsing is immediately aborted and whatever has been parsed so far is returned.
   • The user can choose to throw an error if the state ever exceeds logger_level.
   • Advantages of the new approach:
     1. The parser can return all the valid parts of invalid JSON up to the point where the fatal error occurred.
     2. The plugin can use the status bar to show how severely the JSON deviates from the JSON standard.
     3. Tracking of errors in the parser's lint attribute is now independent of the strictness of the parser.
3. __The allow_comments, allow_unquoted_string, allow_nan_inf, and linting settings have been eliminated.
   • They have been replaced by the logger_level setting, described here

Minor changes

1. Changed the default value of minimal_whitespace_compression to true.
2. Made it so that automatic parsing and schema validation is off by default.

Added

1. New pretty-printing mode, PPrint.
2. Find/replace form now automatically refreshes tree view on use, to ensure most up-to-date JSON is used
3. Slight improvement to parsing performance, major improvement to pretty-print/compression performance
4. Support for minLength and maxLength keywords in JSON Schema validation of strings.
5. Support for the rest of the JSON5 specification, with the following exception(s):
   • Escaped newlines in strings are ignored. Note that this will not work if you are using newlines other than \r, \n, or \r\n!
   • Escaped digits are simply treated as digits, no matter what.
6. Support for the undefined and None literals, which are parsed as null.
7. Support for the True and False literals. Since None is now also supported, Python-style JSON documents are now fully supported.

Fixed

1. Remove annoying bug where SOH characters (Ascii code \x01) were sometimes added to the end of the document when pretty-printing or compressing.
2. Comments immediately after numbers no longer throw an error
3. Empty unclosed arrays and objects no longer throw an error
4. Paths to treenodes including an empty string key (e.g., {"": 1}) no longer throw an error
5. Better handling of comments, especially empty comments
6. Performance bug (introduced in 4.14.0) in Expand/Collapse all subtreees tree node right-click menu option
7. Better RemesPath error message when user tries to use a function that doesn't exist
8. RemesPath function names can be used unquoted when they're not being called as functions. For example, @.items would have raised an error previously because items is a function name, but now there's no problem.