Pretty printing JSON shows incorrect values for big numbers

[a85]

For example,

Raw:

[{"id":89367537255224593},
{"id":89367537255224594},
{"id":89367537255255009},
{"id":89367537255255010}]

Pretty printed:

[
{
"id": 89367537255224590
},
{
"id": 89367537255224590
},
{
"id": 89367537255255000
},
{
"id": 89367537255255000
}
]

I am using vkbeautify with Postman and this bug was submitted here. https://github.com/a85/POSTMan-Chrome-Extension/issues/316. Would be great to see this fixed.

[vkiryukhin]

Well... bad news: it is kernel level issue and cannot be fixed without complete redesigning of the vkbeautify.json() part of the plugin. The matter is: the values exceed the range of "integral precision". They are longer than 16 digits. Current version of the vkbeautify plugin uses native JSON object to beautify text. And when we process these long data with JSON object, some of the least-significant bits are lost.

Now good news: originally, json beautifying was done with regexp technique, which doesn't have "precision" restrictions. I think I can restore this version. But I don't have time right now, so it'll take one or two weeks. If it's urgent, you can do it yourself. All changes are in git. Clone it and customize as you need. Or, wait a little, and I'll do it for you.

Best regards,

--Vadim

[a85]

Ah. Was thinking the same but rather weird that the native JSON object would fail at big numbers. The next release of Postman is still some time away so if you can do it in a week or two that would be great. :)

[a85]

Hi,

Was just wondering if you got the time to look into this. :)

Thanks!

[vkiryukhin]

Not yet. Maybe I work on this on weekend. One question to you: do you pass argument as an object or as a text? If text, it's fine. If object, regexp approach will not help, as I need to convert object in text first, which brings the same "precision restriction" issue.

[a85]

Thanks. I pass the argument as text.

[vkiryukhin]

Hi Abhinav,

I've implemented regexp version on json and created json_regexp branch. Please take a look. It would be nice if you test it in your environment. I did brief testing, everything looked OK. But another round of testing would be useful.

Thank you,

--Vadim

[a85]

Awesome! Thanks. I will run some tests here and update the issue.

[a85]

Just tested. The big number issue is gone but a couple of things are different:

1. Need to give an extra parameter to the json function
2. Braces start from a new line. That is instead of

form: {
}

it is pretty printed as

form:
{
}

This makes the string rather big to glance through. Would be great if there was an option to toggle it's behavior

[vkiryukhin]

Hey Abhinav,

I've fixed the bug and improved formatting "key : {"

Please re-test it.

One known problem: it formats content inside a quoted string. For example, value part in the statement "key": "abc : [ 1,2,3 ]" will be formatted. So, if your users get complex stuff like regexp as a string value, it can be issue. If it is important, please let me know. Everything else looks OK.

--Vadim

[a85]

Hey Vadim,

The big int numbers bug is gone and the formatting looks better. The string issue does seem to be a problem though. The commas get split into new lines and that makes the JSON string look buggy. Would be good to have that as v0.98.

[vkiryukhin]

Hey Abhinav,

I've fixed "comma in string" issue, as well as other unwanted in-string formatting cases. Please test it in your environment. Not sure what you meant, asking for v0.98. Do you want I set this particular number as the version? Current version I set is "1.1.00.regexp", but I can set anything :) Just let me know. Thank you,

--Vadim

[a85]

Thanks Vadim! Testing it out. I meant the comma issue to work as version 0.98 which I was using. That's all. :)

[a85]

Seeing slightly weird behavior. The string below was pretty printed like this:

{
    "origin": "127.0.0.1",
    "files": {
    },
    "form": {
    },
    "url": "http: //localhost: 5000/post",
    "args": {
    },
    "headers": {
        "Origin": "chrome-extension: //aglpidefogoeiamaehklpfoafichfmdk",
        "Content-Length": "29",
        "Accept-Language": "en-US,
        en;q=0.8,
        pt;q=0.6,
        af;q=0.4,
        mo;q=0.2,
        zh;q=0.2",
        "Accept-Encoding": "gzip,
        deflate,
        sdch",
        "Host": "localhost: 5000",
        "Accept": "*/*",
        "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML,
         like Gecko) Chrome/30.0.1599.10 Safari/537.36",
        "Connection": "keep-alive",
        "Cache-Control": "no-cache",
        "Content-Type": "application/json",
        "Postman-Token": "ebaf1ba5-a819-1655-263b-f16d11d1eed6"
    },
    "json": {
        "json": "is quite cool"
    },
    "data": "{
        \n  \"json\":  \"is quite cool\"\n}"
    }

However with some responses the "Accept-Language" key was not split into different lines.

[vkiryukhin]

can you please send me the data before formatting?

[a85]

Yep. Here it is. This is formatted already from the server side but the same kind of response might be minified too.

{
  "origin": "127.0.0.1",
  "files": {},
  "form": {},
  "url": "http://localhost:5000/post",
  "args": {},
  "headers": {
    "Origin": "chrome-extension://aglpidefogoeiamaehklpfoafichfmdk",
    "Content-Length": "31",
    "Accept-Language": "en-US,en;q=0.8,pt;q=0.6,af;q=0.4,mo;q=0.2,zh;q=0.2",
    "Accept-Encoding": "gzip,deflate,sdch",
    "Host": "localhost:5000",
    "Accept": "*/*",
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.10 Safari/537.36",
    "Connection": "keep-alive",
    "Cache-Control": "no-cache",
    "Content-Type": "application/xml",
    "Postman-Token": "fab24b79-fe62-09dd-f573-bd435d5fc09d"
  },
  "json": null,
  "data": "<html>\n  XML not so much\n</html"
}

[vkiryukhin]

well... this stupid '\n' breaks my smart regexp construction :) Gonna to try completely different approach.

[vkiryukhin]

OK, finally, this version works as expected. I refused regexp technique, as it is unreliable and use "template-style" approach. In this version I replace long (> 16 chars) numbers with valid delimiter and let JSON to beautify text, then take formatted string and replace delimiters with original numbers. That's it :) Please test it.

[a85]

Awesome. Seems to be working well!