Closed kadler closed 6 years ago
Original comment by Brian Jerome (Bitbucket: bjerome, GitHub: brianmjerome).
Will do. Thanks!
Edit This is looking better with 1.1.3-sg3.
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
update db2procjr (laptop)
You may have been using db2procj or db2projr. Escapes same issues as Jess G. Re-try your test with 1.1.2-sg9.
Original comment by Brian Jerome (Bitbucket: bjerome, GitHub: brianmjerome).
I'll have a discussion about that. Running JSON.parse() (in JS) on the content of outone gets an error so we need that extra escape in there. I'll mark this resolved for now as it looks good. Thanks.
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
Am I overthinking this?
Argh ... escape logic same effect looking into a mirror with a mirror behind you, and you, and you , and you, one you, two you, three you, big you, little you, mini you, ... argh!!!
JSON validator liked both of following escape sequences.
#!bash
loves it ... \"<span class=\\\"subheading\\\">Subhead</span>\", ...
== and ==
loves it ... \"<span class=\"subheading\">Subhead</span>\", ...
full json ...
#!bash
to escape ...
{
"script": [{
"pgm": ["DANNY03", "DB2JSON", {
"outone": "{\"title\":\"Hello World\",\"content\":\"<span class=\\\"subheading\\\">Subhead</span>\",\"footer\":{\"links\":[\"www.google.com\",\"www.mysite.co\"],\"height\":50}}"
}]
}]
}
or not to escape ...
{
"script": [{
"pgm": ["DANNY03", "DB2JSON", {
"outone": "{\"title\":\"Hello World\",\"content\":\"<span class=\"subheading\">Subhead</span>\",\"footer\":{\"links\":[\"www.google.com\",\"www.mysite.co\"],\"height\":50}}"
}]
}]
}
I don't know. Maybe just best to assume that all output is never 'escaped'. Aka, you go argue with your RPG guys about not pre-escaping json before the toolkit????
Original comment by Brian Jerome (Bitbucket: bjerome, GitHub: brianmjerome).
I'm not understanding what you mean by partial escape. If the JSON value (which is already a string) contains a JSON string then shouldn't the \ also be escaped to \\ in the output? I figured otherwise there would be an error from invalid JSON. Am I overthinking this?
#!json
...\"<span class=\"subheading\">Subhead</span>\"...
#!json
...\"<span class=\\\"subheading\\\">Subhead</span>\"...
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
I have not fixed this yet (other Rochester lab work). In db2sock json-parser, we should attempt to 'discover' if json data is already escaped ...
Ok. I think this fix may work for none, full or partial escaped json "value" (aka, your RPG guys pre-escaping some/all json ... argh).
#!bash
=============================
test1000_sql400json64 ../json/j0911_pgm_danny03_escape
=============================
input(5000000):
{"pgm":[
{"name":"DANNY03", "lib":"DB2JSON"},
{"s": [
{"name":"outone", "type":"64000a", "by":"out"},
{"name":"flag", "type":"10i0", "value":0, "by":"in"}
]}
]}
output(217):
{"script":[{"pgm":["DANNY03","DB2JSON",{"outone":"{\"title\":\"Hello World\",\"content\":\"<span class=\"subheading\">Subhead</span>\",\"footer\":{\"links\":[\"www.google.com\",\"www.mysite.co\"],\"height\":50}}"}]}]}
result:
success (0)
=============================
test1000_sql400json64 ../json/j0912_pgm_danny03_escape_partial
=============================
input(5000000):
{"pgm":[
{"name":"DANNY03", "lib":"DB2JSON"},
{"s": [
{"name":"outone", "type":"64000a", "by":"out"},
{"name":"flag", "type":"10i0", "value":1, "by":"in"}
]}
]}
output(217):
{"script":[{"pgm":["DANNY03","DB2JSON",{"outone":"{\"title\":\"Hello World\",\"content\":\"<span class=\"subheading\">Subhead</span>\",\"footer\":{\"links\":[\"www.google.com\",\"www.mysite.co\"],\"height\":50}}"}]}]}
result:
success (0)
RPG danny03
#!c
dcl-pi Main;
outone char(64000);
flag int(10);
end-pi;
// partial escaped json ... argh (subheading)
if flag = 1;
outone = '{"title":"Hello World","content":"<span ' +
'class=\"subheading\">Subhead</span>","footer":' +
'{"links":["www.google.com","www.mysite.co"],"height":50}}';
// not escaped json (yes)
else;
outone = '{"title":"Hello World","content":"<span ' +
'class="subheading">Subhead</span>","footer":' +
'{"links":["www.google.com","www.mysite.co"],"height":50}}';
endif;
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
CALL DB2JSON.DB2PROCJR
I repeat. DB2PROCJR is not a good interface to build a toolkit, only used for remote use laptops, etc.. You should use direct call SQL400Json and/or SQL400JsonAsync.
#!c
SQLRETURN SQL400Json( SQLHDBC hdbc,
SQLCHAR * injson, SQLINTEGER inlen,
SQLCHAR * outjson, SQLINTEGER outlen );
pthread_t SQL400JsonAsync ( SQLHDBC hdbc,
SQLCHAR * injson, SQLINTEGER inlen,
SQLCHAR * outjson, SQLINTEGER outlen,
void * callback );
Expanding to be clear ...
Assuming your new toolkit is targeting node. If your node toolkit is using REST (fastcgi), then you will get natural async behaviour (good node scripts). However, using CALL DB2JSON.DB2PROCJR is not a good idea for a db2 interface for your node toolkit (aka, not async at all). You need to modify the node db2 driver to support SQL400Json/SQL400JsonAsync to build a good db2 interface (aka, change db2a c code for node/npm). If this is beyond your skill level, you should ask for help.
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
Updated to sg5..
I have not fixed this yet (other Rochester lab work). In db2sock json-parser, we should attempt to 'discover' if json data is already escaped to avoid double slash-quote error (distorted). I suspect we will be able to sort this out when i find time to work on this problem.
Original comment by Brian Jerome (Bitbucket: bjerome, GitHub: brianmjerome).
Updated to sg5..
I tried running danny03 test but that has the same weird result as mine. Something else must be wrong here.
#!sql
CALL
DB2JSON.DB2PROCJR('{"pgm":[
{"name":"DANNY03", "lib":"DB2JSON"},
{"s": {"name":"outone", "type":"64000a", "by":"out"}}
]}');
#!json
{"script":[{"pgm":["DANNY03","DB2JSON",{"outone":"{\"title\"\"\"HellloWorld\",\"\"oontent:\"<span lass=\"ubheadinnn\">Subbhd</spannn\",\"ffoor\"::\"liiikk\"\":[\"\"wggoggll..oom\",,w..yssiieecco\"\"\"hightt\"\"0`}"}]}]}
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
Delay. Working on unrelated PASE lab issue. Will let you know when avail here.
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
Cool. i need to do some more work anyway (below).
Note to self (geek reminder - myself) ... i should move json specific formating out of base toolkit. Dealing with json escape should be contained in toolkit/parser-json, not in base toolkit ILE converters. We have not yet added other parser formats like xml, cvs, but probably will some date.
Original comment by Brian Jerome (Bitbucket: bjerome, GitHub: brianmjerome).
I'm thinking the data might have been escaped for Java, but I'll have to discuss that at another time when they are available. I will try getting the latest driver when I have some time. Probably won't be able to until Monday unfortunately (same goes for the other inputDS issue)
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
Continued ... Just realized I am not handling escape json input correctly.
Please look into previous suggested tests. We need to understand if RPG already escaped your output (we do it twice then).
Monday I will look into possible ways of automatically detecting already escaped json both input and output.
Thanks for your help.
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
I have few ideas for you.
First, try calling 1.1.2-sg4, danny03. Let's try to see if escape works right taking your RPG out of the equation.
Second, Let's go back to you original statement use to work before escape sequences added to db2sock. Possible test your RPG guy (not you), escaped the json data for another use ( Java , etc). Therein we are not getting the full story here (already escaped data).
If this is the case, we need to add flag like "s" like "no escape":"on".
Again, I do not say to offend. I am merely trying to help. Your other closed issue your RPG guy did not give you the real data layout. So, well, we maybe should assume RPG guy is messing with you again. Not to worry, we will get it working even if your RPG guys make it difficult.
Btw - sorry about the typos, my bloody iPad keeps substituting wrong words while I am typing (big help it be .... Not).
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
Mmm ... same test works for me (posted above). Maybe try moving up a few releases to test driver - 1.1.2-sg4? If still an issue, i will have to figure out why mine works and you does not.
Original comment by Brian Jerome (Bitbucket: bjerome, GitHub: brianmjerome).
If it has class="subheading" then it's not a valid JSON string.
I took that out completely to avoid that being an "issue" but it is still distorted in sg10.
outone = '{"title":"Hello World","content":"<span ' +
'>Subhead</span>","footer":' +
'{"links":["www.google.com","www.mysite.co"],"height":50}}';
#!json
{"script":[{"pgm":["RETURNJSON","BJEROME",{"hello":"{\"title\"\"\"HellloWorld\",\"\"oontent:\"<span Subhea</span>\"\"\"\"fooot\":{\"\"inkkkk\"[[\"ww.ooole....o\"\",\"\"wmyyiteeeec\"\",\"\"eght\":55`"}]}]}
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
BTW -- Please close issues when satisfied with answer/result. No need to clutter issues database with finished items. I always forget and have to read through the last chat each week. Thanks
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
user error ... i think
Looks like your RPG program has a 'half-escaped' return.
#!c
outone = '{"title":"Hello World","content":"<span' +
'class=\"subheading\">Subhead</span>","footer":' +
'{"links":["www.google.com","www.mysite.co"],"height":50}}
-- should be ---
outone = '{"title":"Hello World","content":"<span' +
'class="subheading">Subhead</span>","footer":' +
'{"links":["www.google.com","www.mysite.co"],"height":50}}
Note: class=\"subheading\" should be class="subheading" (no escape)
I corrected your RPG in my program ... and works.
#!c
H AlwNull(*UsrCtl)
dcl-pr Main extpgm;
outone char(64000);
end-pr;
dcl-pi Main;
outone char(64000);
end-pi;
outone = '{"title":"Hello World","content":"<span ' +
'class="subheading">Subhead</span>","footer":' +
'{"links":["www.google.com","www.mysite.co"],"height":50}}';
return;
Note (minor): You also missed space between '<span class'.
The run ...
#!bash
$ ./test1000_sql400json32 ../json/j0911_pgm_danny03_escape
input(5000000):
{"pgm":[
{"name":"DANNY03", "lib":"DB2JSON"},
{"s": {"name":"outone", "type":"64000a", "by":"out"}}
]}
output(217):
{"script":[{"pgm":["DANNY03","DB2JSON",
{"outone":"{\"title\":\"Hello World\",
\"content\":\"<span class=\"subheading\">Subhead</span>\",
\"footer\":{\"links\":[\"www.google.com\",\"www.mysite.co\"],
\"height\":50}}"}]}]}
result:
success (0)
Suggestion ... arrogant not intended ... assume toolkit is correct and error is test case (first). Aka, close look at your test. I am only working part time Mon-Wed, so things could sit unanswered every week.
Note: I have extra line feeds added to above output to fit within small box of this issues output text. You need to remove extra line feeds to validate json cut/paste from here (obvious???).
Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).
DB2JSON.DB2PROCJR
First, DB2PROCJR uses an ILE port of PASE libtkit400.a function. So, well, ccsid hell, not recommended interface for anything production beyond remote access via ODBC/CLI LUW laptops.
Just a warning ... mmm ... 'worked before' escape sequences maybe indicates not using a client side json parser/verifier on return output. Aka, if building an intermediate 'middleware chunk' (language toolkit), you may be passing 'invalid json' buck-stops-here to your callers/users.
What I get is the output distorted in a weird way
However, generally, we should be able to wade through json escape sequences correctly. In this case you have a whole lot of them (Uf Da).
I will take a look.
Original report by Brian Jerome (Bitbucket: bjerome, GitHub: brianmjerome).
I have a simple pgm returning a JSON string:
And I'm calling the PGM with the following with the sg10 build:
I'm expecting it to return something like:
What I get is the output distorted in a weird way:
Might this be related to the change in sg8 to escape quotes in strings? I ran this code with sg6 and the text was fine but not escaped.