Differences between JsonPreprocessor and Python JSON interface

[HolQue]

!! Please click on EDIT when you read this issue. The text inside contains a lot of backslashes and I have not masked them (I would become mad if I hat to do this). But GitHub resolves them. Therefore the rendered text looks different !!

W.r.t. the recently introduced naming convention check the JsonPreprocessor results differ from results of the Python JSON interface in some cases.

Is this wanted? Please valuate the following usecases or at least be aware of the differences.

I pass the same expression to the JsonPreprocessor and also - within a separate Python script - to json.load(<JSON file>).

Background of every following code example is what I found in internet about valid JSON key names:

(a) Keys in object can be any string, and the production for string says that it can contain any Unicode character, as well as various escape sequences. The specification for the backslash character in strings says that it can only be followed by certain characters that form designated escape sequences.

(b) The following characters must be escaped in JSON data to avoid any problems: " (double quote) \ (backslash) all control characters like \n, \t

(c) Unicode codepoints U+D800 to U+DFFF must be avoided: they are invalid in Unicode because they are reserved for UTF-16 surrogate pairs.

(1)

"A\B" : "value" (violates (b))

JsonPreprocessor: Invalid key name detected: '"A\B"' Python load(): Invalid \escape: line ...

(what is the reason for '\' renamed to '\' in error message?)

(2)

"A\\B" : "value"

JsonPreprocessor: Invalid key name detected: '"A\B"' Python load(): {'A\B': 'value'}

Deviation!

(3)

"A\nB" : "value"

JsonPreprocessor: Invalid key name detected: '"A\nB"' Python load(): {'A\nB': 'value'}

Deviation!

(4)

"A\\nB" : "value"

JsonPreprocessor: Invalid key name detected: '"A\nB"' Python load(): {'A\nB': 'value'}

Deviation!

(5)

"A\"B" : "value"

JsonPreprocessor: {'A"B': 'value'} Python load(): {'A"B': 'value'}

(6)

"para\u0020m" : "value" (accordingly to (a): unicode coded blank)

JsonPreprocessor: {'para m': 'value'} Python load(): {'para m': 'value'}

(7)

"param\-" : "value" (violates (a))

JsonPreprocessor: Invalid key name detected: '"param\-"' Python load(): Invalid \escape: line ...

(what is the reason for '-' renamed to '\-' in error message?)

(8)

"param\x" : "value" (notation incomplete)

JsonPreprocessor: Error: ''unicodeescape' codec can't decode bytes in position 6-7: truncated \xXX escape'! Python load(): Invalid \escape: line

The different wording of the error messages astonishes me.

(9)

"param\1" : "value"

JsonPreprocessor: Invalid key name detected: '"param\x01"' Python load(): Invalid \escape: line ...

(what is the reason for '\1' renamed to '\x01' in error message?)

(10)

"param\\1" : "value"

JsonPreprocessor: Invalid key name detected: '"param\1"' Python load(): {'param\1': 'value'}

Deviation!

(11)

"param\uD800" : "value" (violates (c))

JsonPreprocessor: UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position 30: surrogates not allowed Python load(): UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position 34: surrogates not allowed

JsonPreprocessor crashes! Missing try/except block? This should be fixed.

(12)

"1*~@#$%^&*(')_+=><?/\"ÄÖÜ.ß.€.考.𠼭.𠼭": "value"

JsonPreprocessor: {'1~@#$%^&(\')+=><?/"ÄÖÜ.ß.€.考.𠼭.𠼭': 'value'} Python load(): {'1~@#$%^&(\')+=><?/"ÄÖÜ.ß.€.考.𠼭.𠼭': 'value'}

No deviation!

[test-fullautomation]

Hi Holger, thank you for the detailed test. For me no show-stopper inside => shifted to 0.12.0 Thomas

[namsonx]

Hello Holger,

Thank you for your comments, I pushed the small change to align the JsonPreprocessor and Python JSON interface to stabi branch.

Thank you, Son

[HolQue]

Retest successful. Issue can be closed.