enable support for JSON data types other than string

[mrkazoodle]

Hi Henk-Jan,

In most output types everything needs to be converted to text, but for JSON output, in order to be able to write true and false values for example, not every value may be converted to text. The solution of these changes: a putraw() function which allows these values to be passed on to the JSON writer.

[eppye-bots]

I am doing json improvements. is this only for datatype 'bool' ? (not for int and reals (which in json do not have quotes)?

[mrkazoodle]

After quickly going through the code: no, not just booleans.

Booleans were missing as a data type, so I added it. Let me double check and get back on this. (+ I'm afraid I've spotted an issue with checking if the (formatted) numeric value passed to outmessage is a string: it only accepts/checks unicode strings, instead of all types of strings)

Putraw() is generic: it is a copy paste of put() that skips the conversion to unicode strings.

This part is write only (order intake from edifact), but I have started reading JSON as well (invoices & desadv), but that is newer and not yet as clean.

Let me get back on this: Rome is niet in 1 dag gebouwd :-)

[mrkazoodle]

Good Morning Henk-Jan,

We have these patches in production for a couple of months: orders are translated to JSON in house format with a boolean and a numeric field (integer more specific).

So, I can confirm this does not only do booleans, but numerics as well. This is all you need for writing to JSON files.

Regarding the input of JSON files: I've looked into my code, and I'm not happy with it, so I can't share it at this point. I will share my experience though: the JSON values are parsed and subsequently cast to strings, so that numerics lose precision and their formatting cannot be validated, and booleans change from true to 'True' (string). In an ideal scenario, the parser should preserve the data as a numeric or boolean, so that getdecimal() doesn't have to convert it back, and that we can have a getboolean() that doesn't return (value == 'True'). If this isn't urgent, I'm glad to help, but you'd have to point me in the direction of where the parsing happens.

[eppye-bots]

json is text......that is the essence of edi. a 'float' or 'int' lives in your computer, and needs to be converted to text in edi. there is no way to avoid this stategy of bots is to avoid this by using decimal library (which does not have the rounding issues float have). I have a implementation now, which is much inspired by your code. Testing this now.

[eppye-bots]

I have to admit: the whole idea of 'intergers' or 'float' is json is pretty much a fake concept to me. Json is just text.

[mrkazoodle]

HJ,

I just mean it takes text (JSON), parses it to whatever numeric type, and casts it back to a string, while losing formatting. This is a lot of conversions...

[eppye-bots]

int is probably not a problem, floats are. there are rounding issues with floats. I started building json into bots without supporting specific 'float' type. bots uses python json library. that does (for incoming json) conversion: text -> python floats think only decent way out out this is to force json library to use Decimal of text....

[mrkazoodle]

Sscc codes are also a problem: they are too long, and also lose precision because of casting to float... (General issue I believe, I now treat them as alphanumeric because of this issue)