hydrobyte / McJSON

A Delphi / Lazarus / C++Builder simple and small class for fast JSON parsing.
MIT License
58 stars 20 forks source link

Shorteners self-test result #7

Closed totyaxy closed 1 year ago

totyaxy commented 1 year ago

Hi!

It's not a bugreport, only help for you (with "," separator):

[PASS] Test 01: parse simple object [PASS] Test 02: parse simple array [PASS] Test 03: parse simple sub object array [PASS] Test 04: simple object value change [PASS] Test 05: Add, Insert, Delete functions [PASS] Test 06: object is nil Error: Invalid index: get item by index 3 [PASS] Test 07: getters and setters [FAIL] Test 08: numbers: scientific notation Error: Can't convert item "number" with value "-1.23456789E-10" to "double" [PASS] Test 09: escapes [PASS] Test 10: invalid JSON [PASS] Test 11: valid or unusual JSON [FAIL] Test 12: type transformations [FAIL] Test 13: Save and Load using files [PASS] Test 14: constructors [PASS] Test 15: Copy, Clone, IsEqual, Remove functions [PASS] Test 16: exceptions Error: Object reference is nil: get item by key "not" Error: Object reference is nil: get item by key "not" Error: Object reference is nil: get item by index 1 Error: Invalid item type: expected "object" got "value" Error: Can't convert item "string" with value "123a" to "integer" Error: Can't convert item "null" to "integer" Error: Error while parsing text: "duplicated key k" at pos "14" [PASS] Test 17: enumerators [FAIL] Test 18: example like JsonDataObjects [PASS] Test 19: At() shortener for array item access [PASS] Test 20: key paths [FAIL] Test: Github readme.md content Error: Error while parsing text: "expected : got 3" at pos "48"

5 tests FAILED

With "." separator:

[PASS] Test 01: parse simple object [PASS] Test 02: parse simple array [PASS] Test 03: parse simple sub object array [PASS] Test 04: simple object value change [PASS] Test 05: Add, Insert, Delete functions [PASS] Test 06: object is nil Error: Invalid index: get item by index 3 [PASS] Test 07: getters and setters [PASS] Test 08: numbers: scientific notation [PASS] Test 09: escapes [PASS] Test 10: invalid JSON [PASS] Test 11: valid or unusual JSON [PASS] Test 12: type transformations [FAIL] Test 13: Save and Load using files [PASS] Test 14: constructors [PASS] Test 15: Copy, Clone, IsEqual, Remove functions [PASS] Test 16: exceptions Error: Object reference is nil: get item by key "not" Error: Object reference is nil: get item by key "not" Error: Object reference is nil: get item by index 1 Error: Invalid item type: expected "object" got "value" Error: Can't convert item "string" with value "123a" to "integer" Error: Can't convert item "null" to "integer" Error: Error while parsing text: "duplicated key k" at pos "14" [PASS] Test 17: enumerators [PASS] Test 18: example like JsonDataObjects [PASS] Test 19: At() shortener for array item access [PASS] Test 20: key paths [PASS] Test: Github readme.md content

1 tests FAILED

Lazarus 2.2.4 x64 (official installer) on Win10 x64

hydrobyte commented 1 year ago

Hi,

Please, test the "main" branch for decimal separator awareness and other fixes. See InternalFloatToStr and InternalStrToFloat.

The "Shorteners" branch will be merged to "main" soon.

Regards,

totyaxy commented 1 year ago

I thought the shortener was a newer version. The trunk version with "," or "." only 1 test failed (save and load using files).

hydrobyte commented 1 year ago

Nice. Could you debug it and report why this test failed? Here I have two Windows PCs with Lazarus 2.2.0 and 2.3.0 and both have zero failures.

totyaxy commented 1 year ago

Hi, yes.

If any of them allowed then reasult test failed:

// load a Ansi file (especifying it)
M.LoadFromFile('test13-Ansi.json', false);
Result := Result and (M['ansi'].AsString = 'ăçüö');

// load a UTF-8 file
M.LoadFromFile('test13-UTF8.json');
Result := Result and (M['utf8'].AsString = 'ăçüö');

But not a surprise, as I see {$mode delphi} , and {$codepage cp1252} on the file header.

If I see these file with notepad++

  1. test13-Ansi.json (force cp1252 codepage because I see garbage otherwise): {"ansi":"ãçüö"} (it's different as you see)

  2. test13-UTF8.json (auto UTF8 detected correctly): {"utf8":"ãçüö"} (it's different too as you see)

Then I try to rewrite the "ă" to "ã" in the source, then I got message from the Lazarus: "save file cp1250 loose characters.."

So this is character encoding problem.

hydrobyte commented 1 year ago

Hi,

  1. "test13-Ansi.json" is a ANSI file with CP-1252 characters. Here in my Notepad++ (via menu Encoding > Character sets > Western European > Windows-1252) I can see "{"ansi":"ãçüö"}" as expected.

  2. I've set {$codepage cp1252} in PrjTestMcJSON.lpr because the .lpr file is not encoded in UTF-8. It opens correctly, it compiles and run OK.

  3. Into your description I just see "ãçüö". There is no "ă" (items 1 and 2).

I think this error is related with your Windows (System) Codepage, that might be different from 1252. I just don't know why {$codepage cp1252} failed in order to prevent this CodePage misleading.

I'll explore changing my source code files to UTF-8, but I think this will broke Delphi (old versions) and Lazarus/FPC compatibility. I'll try changing just the .lpr file first to UTF-8 and improve Test13.

On the other hand, I think this error can't stop you from using McJSON if you work with .json files encoded in UTF-8.

Thanks for sharing. Stay tuned to next commits.

Regards,

totyaxy commented 1 year ago

Hi, I stayed, with the latest revision 94, the result of test 13 is still failed (Laz 2.2.4 and Laz 2.3.0 both same results)

hydrobyte commented 1 year ago

Hi,

I was revising Lazarus support to UTF-8, comparing it to Delphi pre-Unicode, and I think it is expected that Test13 fails with a Windows CodePage different from 1252.

I'll close this thread because it is in the wrong branch.

Please, create a new Issue within main branch and report your Windows CodePage.

Regards,