Get Involved

[kadler]

Original report by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

Would like to contribute

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Ok fixed project as400_types.h (PASE include). Also recompiled test version on YIPs test driver

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Opps ... something wrong new compile ... will fix.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Ok, changed everything to chroot /QOpenSys/db2sock - see source README(s)

BTW -- This is not ready for big test yet. Needs bit more completed in the json 'toolkit' area to get a good feel for how works. However, you may try anything you like.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Also, i just notice i am using same chroot /QOpenSys/zend7 as php 7. I will change my project to chroot /QOpenSys/db2sock and update any reference in this git project to 'db2sock'. Oversight on my part.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Ok, did not want to be a Zend Server discussion. Assuming php7 (you). I install as production to turn off developer helper features. Most particular stress to CPU was turn off development Z-Ray.

Z Ray on/off

[kadler]

Original comment by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

OK I have some issues, I installed the Zend stack and have a number of issues which are making it difficult to work with (CPU utilization through the roof and no admin access to change the settings). I have logged a couple of requests on the forums so hope to get some responses. I am still waiting for some info on how to build a stack from source so even that's not where I would like it to be. Hope to get something we can work with soon :-)

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Well, does not really matter which language. However, some other guys joining the 'toolkit' party are php people. Wherein, php is a common language.

Also, no 'public' access to nodejs db2 driver and python db2 driver (not open source), so i hate to involve 'testing stuff' that is locked behind IBM. Aka, i want everyone to understand exactly how the PASE c code works and interacts with the ILE side.

So, php is really best, everything is in the open (open source).

[kadler]

Original comment by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

Tony

Still trying to get a suitable PHP based solution for testing, do you think it would be better to test with Node.js? I may just bite the bullet and install Zend PHP for now if not?

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Next week. Current design of JSON input "fairly" minimalist. Of course, require a manual, toolkit wrapper, so on, but general idea.

#!c

/* json
 * request {
 * -- toolkit database --
 * "query":"select * from bobdata",
 *   "fetch":"*ALL",
 * "query":"call proc(?,?,?)",
 *   "parm":[1,2,"bob"],
 * "connect":["*LOCAL","UID","PWD"],
 *   "query":"call proc(?,?,?)",
 *   "parm":[1,2,"bob"],
 *   "fetch":"*ALL",
 * "pconnect":["id"],
 *   "query":"select * from davedata where name=? and level=? and reports=?",
 *   "parm":[1,2,"bob"],
 *   "fetch":"*ALL",
 * -- toolkit commmand --
 * "cmd":"ADDLIBLE LIB(DB2JSON)",
 * -- toolkit program --
 * "pgm":["NAME","LIB","procedure"],
 *   "dcl-ds":["name",dimension, "in|out|both|value|const|return", "dou-name"],
 *   "end-ds":"name",
 *   "dcl-s":["name","type", value, dimension, "in|out|both|value|const|return",ccsid],
 * "end-pgm":"NAME",
 * -- complex parm (example)               -- temp_t[] RegionTemps(regions_t,int,int)
 * "pgm":["CLIMATE","MYLIB","RegionTemps"],-- *SRVPGM MYLIB/CLIMATE
 *   "dcl-ds":["regions_t"],               -- ds parm assumed "both" --
 *     "dcl-s":["region","5a","TX"],       -- region[0] = "TX"
 *     "dcl-s":["region","5a","MN"],       -- region[1] = "MN"
 *     "dcl-s":["region","5a","", 20],     -- region[2-21] = "" --
 *     "dcl-ds":["people_t",20],           -- ds[20] nested --
 *       "dcl-s":["first","32a"],
 *       "dcl-s":["last","32a"],
 *     "end-ds":"people_t",
 *   "end-ds":"regions_t",
 * -- single parm --
 *   "dcl-s":["countout","10i0",0,"both"],
 *   "dcl-s":["available","10i0"],         -- assumed "both" (not inside ds) --
 * -- complex return value --
 *   "dcl-ds":["temp_t",999, "return","countout"],
 *     "dcl-s":["region","5a"],
 *     "dcl-s":["min","12p2"],
 *     "dcl-s":["max","12p2"],
 *   "end-ds":"temp_t",
 * "end-pgm":"CLIMATE",
 * }
 * -- types --
 * "5a"    char(5)         char a[5]
 * "5av2"  varchar(5:2)    struct varchar{short,a[5]}
 * "5av4"  varchar(5:4)    struct varchar{int,a[5]}
 * "5b"    binary(5)       char a[5]
 * "5bv2"  varbinary(5:2)  struct varbinary{short,a[5]}
 * "5bv4"  varbinary(5:4)  struct varbinary{int,a[5]}
 * "3i0"   int(3)          int8, char
 * "5i0"   int(5)          int16, short
 * "10i0"  int(10)         int32, int, long
 * "20i0"  int(20)         int64, long long
 * "3u0"   uns(3)          uint8, uchar, char
 * "5u0"   uns(5)          uint16, ushort, unsigned short
 * "10u0"  uns(10)         uint32, uint, unsigned long
 * "20u0"  uns(20)         uint64, ulonglong, unsigned long long
 * "4f"    float           float
 * "8f"    double          double
 * "12p2"  packed(12:2)    (no c equiv)
 * "12s2"  zoned(12:2)     (no c equiv)
 * "8h"    hole            hole
 */

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Ok, i added a PASE fastcgi warpper to libdb400.a for JSON REST.

Now we can configure either ILE CGI or PASE fastcgi. Both appear to work by PHP tests in this project (db2sock/tests_php).

Should be noted that i am 'trying out' ILE CGI persistent. That is, keeping PASE running in the Apache CGI job. Technically, secret sauce for ILE CGI persistent is using a named activation group (DB2JSON activation). Not 100% sure side effects of 'fast' ILE CGI 'persistence', but appears to work.

Both CGI and fastcgi will accept a db2sock persistent connection in the json rest for performance (see tests_php/cgi_querypconnect.php and tests/fastcgi_querypconnect.php respectively). Of course, more 'web performance tune' needed, but gives an idea of how things may work.

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

See ILE-CGI/db2json.c (note: A lot less code over RPG, so i am happy to switch on popular demand.)

No kidding! Love it.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Good shirt.

Now ...

On the project side, i have adjusted to use ILE c. I removed references and code RPG. I only have one ILE part converted so far. See ILE-CGI/db2json.c (note: A lot less code over RPG, so i am happy to switch on popular demand.)

Next ...

Will take me a few days to week to 'tool' my other toolkit work to ILE c. No big deal, just finish designs/code (geek with a keyboard).

I will post a message here when ready for some help testing. Thanks.

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

Aka, you got a geek with a dream and a keyboard.

That's t-shirt worthy. COMMONs18! (see below pic)

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

As they will need the C part for PASE why provide ILE in RPG? Now I need 2 skills to manage/contribute if that is my intention.

So, i talked to my business manager (Jesse). He says, most customers that have WDS will have all compilers (including c). Only a few are expected to take measures to ale cart purchase RPG only.

Decision ...

You guys are the primary crew. I will write in C all the way, both sides (ILE and PASE).

Are binaries being distributed?

So, personally, i think we wait to see how things work before we talk distributing. Anyway, i am not authorized publicly to say about IBM i future plans. You may ask Jessi if you would like.

Disclaimer:

Reminder, while i am acting/contributing with approval of my leadership, i do not speak for IBM in a product capacity. The opinions in these issues are my own. Aka, you got a geek with a dream and a keyboard. Not without influence to be sure, but definitely not the boss.

[kadler]

Original comment by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

Tony,

The question appears to be which language should you write this in?

For me 'C' without any question, for others they may want it in RPG?

Does writing it in RPG mean you will get more community input??? It would reduce my input, but others???

If this was me (being selfish) I would write this in the language of my choice. Most if not all of the people who have an interest in this will JUST download the code and compile it (that makes the language question irrelevant because if it don't compile it aint getting used).

As they will need the C part for PASE why provide ILE in RPG? Now I need 2 skills to manage/contribute if that is my intention.

Chris...

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

Should i do the rest of the project in c code (including ILE)?

Opinion: Yes. The willing-to-contribute C community is significantly larger than the willing-to-contribute RPG community.

2) ILE side -- I thought folks had to give and arm and a leg to buy the c compiler. Yes? Would switching to c code be a violation of trust? Open Source at a huge cost to regular Joe that has RPG compiler?

Are binaries being distributed?

There are a handful of IBM i hosting providers that can be used to compile ILE C at little or zero cost.

[kadler]

Original comment by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

Tony

I have all the compilers, not sure if anyone has RPG without C as it was always shipped together? (I get a license key for WDS not individual compilers?).

As for actual cost, if they are into Open Source they are generally NOT compiling on a production system and probably prefer to use Litmus Spaces or another free IBM i Dev environment for any Open Source activity. (goes back to your previous comment about who owns the intellectual property when developed on your employers system).

I am just about to engage with a User group setting up a sandbox dev environment (kindly donated by a MSP). All of the compilers are available even though most of the users will use RPG? I expect that they will prefer to engage in projects that use RPG in the beginning and with hope will migrate to multi language ILE with time.

Chris...

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

I am going to lunch. Please. if possible respond before i return. I need to get the actual code written for the experiment.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Thanks. However, I don't always use the best tool for the job. In fact, now i am wondering about 'community'. This project community to be precise. We could convert all of db2sock project to c code (ILE and PASE). In truth, c programming will be easier for me. Should i do the rest of the project in c code (including ILE)?

This is an Open Source project, emphasis on 'source', wherein customers/vendors have to compile the code. The big question in my mind ... cost???

1) PASE side -- gcc compiler is free (yeah hard to set-up, but free).

2) ILE side -- I thought folks had to give and arm and a leg to buy the c compiler. Yes? Would switching to c code be a violation of trust? Open Source at a huge cost to regular Joe that has RPG compiler?

Comment?

[kadler]

Original comment by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

Tony,

optional read..

I consider you a developer in the truest sense of the word! Not an RPG programmer :-) I hope that is clear to you? As a developer, you take the best tool for the job and use it to its best advantage and move to the next one when it is a better fit.

Chris..

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Sorry we crossed reply paths. Cool ... please have to the c migrate. More the merrier.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Optional read ...

Forgive my observation, but i write in dozens of languages (better than most). Trained for untold years by IBM in c/C++. However, write one RPG program like XMLSERVICE, IBM i community latches idea RPG is your primary language. At times a really strange community.

[kadler]

Original comment by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

Tony, I am sure your C code is top notch! I will still consider taking the ILE RPG /free stuff and create a C version (just for kicks if nothing else).

Chris

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Mmm ... most of this project is c programming. You can do worse than patterning this PASE db2sock c code. Again, you may have to take my word for the quality of the PASE c code, but, it really is good.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

As for ILE program calls I always wonder why we need the stored procedures? Is there no way to have them invoked directly (with obvious checks and balances which would required some kind of prepare request prior to actual request)?

This is a fantastic observation. However, DB2 does not support dynamic load, activate, call by 'string' (aka, look up, 'NAME', 'LIB'). Second, DB2 supports only primitive type call parameters (integer, packed, numeric/zoned, double, clob, blob, etc.). That is, DB2 does not support any data structures as call parameter(s), no support *SRVPGM return structures, nothing complex (really). These primitive database types fall far shy of 'toolkit' sophisticated nested data structures we find in RPG and Cobol. We are left wanting in the 'pure' DB2 space. I see no horizon to merge the two versions of reality 'simple type database' and 'complex type RPG'.

Hence toolkits. In our case, db2sock Open Source 'experiment' into extended APIs like SQL400Json and beyond. Understood?

[kadler]

Original comment by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

FWIW...

I am not particularly interested in RPG development ( I struggle with keeping up with C skills and all the other languages I dabble in "Getting too old to keep it all in my small brain") Maybe could take the RPG code and see if I could mimic the process in C though? Maybe that helps with your question Aaron (see that as we could collaborate :-))

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

Just a personal side comment... I know RPG fairly well (did it for 12yrs) but don't know C/C++ as well. Would welcome someone writing C for a project lie this that knows the RPG perspective so that RPG folks could use this project as a learning experience.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

How much code has been added by the community to the XMLSERVICE using RPG (/Free or other)?

Not much. XMLSERVICE code from Luca for idle/time out functions (timeout signal). Everything else was more request, usually by email, not git issues (sigh).

If its in C you ' MAY ' attract programmers who are not IBM i specific programmers?

Yes. My heart is with loyal guys of RPG. These folks made IBM i into a great machine.

We struggle to get anyone to do ILE submissions for some reason and those that do are not going to be working at the level you are in terms of OS depth and PASE interaction.

I have always found RPG code is considered company assets. Aka, nobody can give away the 'RPG goods' without getting fired (crude). However, secretly, I feel great pride RPG code on IBM i is considered so valuable. A company asset. RPG person a company asset.

Yes. Ironic. My PASE side loves to give away everything allowed to community (approved my leaders), keeping step with the unavoidable march to 'Open'.

So, bottom line, I am more interested in making sure that RPG folks have something to copy/pattern using PASE new cool stuff (hope db2sock project).

I am not able to comment on the performance difference between RPG and C. Maybe you are right and the speed is not going to be an issue

Yes. I sympathize. You may have to take my word. Ironically, APIs we use are mostly 'c-ish' APIs. Here again, opportunity to demonstrate by example using RPG (free) to 'system' and 'PASE' APIs.

project (slight politics)

I like RPG. Community likes RPG. Cool!

[kadler]

Original comment by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

Tony

How much code has been added by the community to the XMLSERVICE using RPG (/Free or other)?

The fact is building it in RPG will limit the Community activity just because you are going to require RPG programmers to do it. If its in C you ' MAY ' attract programmers who are not IBM i specific programmers? We struggle to get anyone to do ILE submissions for some reason and those that do are not going to be working at the level you are in terms of OS depth and PASE interaction.

I am not able to comment on the performance difference between RPG and C. Maybe you are right and the speed is not going to be an issue.

Chris...

[kadler]

Original comment by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

Certainly does, phew..

As for ILE program calls I always wonder why we need the stored procedures? Is there no way to have them invoked directly (with obvious checks and balances which would required some kind of prepare request prior to actual request)?

I obviously know little of the underlying technology and requirements but seems like it should be possible?

Chris...

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Oh, one last explanation. This one especially for Chris. All of the planned ILE parts will be RPG Free.

Why?

Basically, people have RPG skills and compilers (common). Counter, ILE c skills are limited in the community. We want people to be able to understand at least ILE side RPG interaction with Open Source PASE db2sock driver.

Performance difference between ILE c and ILE RPG for the limited scope of activity needed will be a complete wash. Simply, not worth confusing the non-c people. Instead, hopefully, engaging others skills for unimagined uses with RPG free.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Chris, i am assuming the answers above explain your inquiries about web sockets. I found Aaron's reply very helpful (assuming i understood). Again, i have not used the support (not expert). If missing your line of questioning, please feel free to redirect toward pertinent.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Ah, clearer. Also will answer back to Chris.

I am thinking we start out with json output and add bson next.

Completely sensible.

Websockets is a function of the web server so I don't believe we'd need to implement anything in ILE/PASE and instead we can rely on Apache/Nginx. So at the end of the day we kind of get Websockets for "free" and we just need to teach people how to configure them.

Great. I have not used the support. So, I will accept you explanation. That is, if we can do JSON under an ILE CGI, we get 'web sockets' for free. Therein, Chris, we do not need to do anything for the json experiment, beyond perhaps a test using web sockets.

This is where your XMLSERVICE plugins made good sense (if I am understanding them correctly).

To be clear, XMLSERVICE is completely replaced. That is the JSON API is directly coded into the new super driver (SQL400Json new CLI-like API). If we decide, perhaps add yet another 'API' SQL400Bson, so on. Aka, Open Source project, we allow ourselves the freedom to expand the limited architectures of ODBC/CLI to 'do want we need'.

The only 'ILE code' is for natural interaction with other ILE components.

Example 1: The Apache RPG CGI that can call db2sock SQL400Json API (included)

Example 2: On QSQSRVR side of SQL400Json API, new stored procedure stub to load, activate and directly call ILE programs, etc (not in project yet). Similar thing XMLSERVICE does today, but very limited role to only critical ILE need functions.

Expanding on why need a ILE stored procedure stub (example 2) ...

DB2 does not provide PASE stored procedure support, so we need drop to ILE stored procedures to complete the calls to other ILE objects. (Slight 'lie' DB2 does support PASE Java stored procedures ... but starting Java ... mmm ... we end up back at XMLSERVICE speeds).

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

No - Real time gaming (web socket style). What crazy application are you thinking?

Websockets afford more than just speed. They also offer full-duplex stateful communication. Google's Ian Hickson had this to say:

"Reducing kilobytes of data to 2 bytes…and reducing latency from 150ms to 50ms is far more than marginal. In fact, these two factors alone are enough to make Web Sockets seriously interesting to Google."

I hesitate to mention the above, because as you stated, I can create my own Websocket front-end over db2sock and gain the Websocket advantages. What I can't do is go to bson if the result is already in json. This is where your XMLSERVICE plugins made good sense (if I am understanding them correctly). I am thinking we start out with json output and add bson next.

JDBC does not have anything like db2sock JSON REST idea. I have no understanding of the compare in this question. Can you qualify?

I brought up JDBC because of shared end goal of delivering datasets. Dismiss the comment for now.

Do you know how to code up web sockets in ILE or PASE c?

Websockets is a function of the web server so I don't believe we'd need to implement anything in ILE/PASE and instead we can rely on Apache/Nginx. So at the end of the day we kind of get Websockets for "free" and we just need to teach people how to configure them.

[kadler]

Original comment by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

Tony, I am lost? web sockets in ILE? does that not require an ILE based web server? Am I missing the point here? I can create socket based programs in ILE C but my understanding (could be totally out of the park) is that web sockets allow browser to web server connectivity??? Having the ability to call an ILE socket is not a problem (except the EBCDIC to ASCII translations obviously) so having a browser based socket call should be possible and I think I could do that? But is that a web socket??

Sorry if I am missing something here?

Chris...

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

"better than" XMLSERVICE.

Yes. This is the goal of this experiment. Even if we fall short of 'wildly fast", we can do better i am sure of it. Again, we are trying to reach the 80-90% cases, not necessarily the crazy end of demands. However, if we figure out 'crazy' that would be fine (eye of beholder).

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

topic ... future, all database requests decoupled (possible)

I wonder if web sockets allow binary data transfer, aka, bson (binary json). I know many new databases use the binary protocols to avoid unwanted translations, etc. (JSON not 'cut it'). While web sockets with bson would be limiting in flexibilty, assuming binary on web socket ok, may be a very good high speed 'database request' decoupler for scripting languages. aka, 'cut it'.

Do you know how to code up web sockets in ILE or PASE c?

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

@rangercairns, I agree on us living in a REST world. If there's one thing Amazon/Bezos has taught us is that web APIs are the new norm for decoupled success.

That being said, there's a reason Google pushed hard for HTML5's WebSockets and is entertaining things like protocol buffers. The number of AJAX requests from browser to server has probably increased ten-fold in the past few years. My point is we need to make sure we're designing for tomorrows workloads so we're not immediately irrelevant.

With all that said, I am fine with "good enough" in the evolutionary sense of having something "better than" XMLSERVICE. My comments aren't meant to impede current work but instead to flavor approaches/features for the long term.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Would be great if this db2sock could do WebSockets for remote DB2 access from LUW. Will normal HTTP cut it?

Again, we are doing Open Source, so you can decide 'cut it'. However, Tuesday morning speculation before work, with no data, and assuming we are still talking about JSON over REST.

Yes - HTTP should be fine for normal few records (normal ajax style web pages).

No - Real time gaming (web socket style). What crazy application are you thinking?

SO, not such a fine point, if customer application interaction is expected crazy fast in the realm of gaming speeds, massive data record transfers, 'crazy stuff' for lack of better term, then JSON REST will never 'cut it'.

What type of throughput are you expecting when compared to JDBC?

JDBC does not have anything like db2sock JSON REST idea. I have no understanding of the compare in this question. Can you qualify?

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

We appeared out of order.

The former XMLSERVICE would have been too slow for frequent database access. Would be great if this db2sock could do WebSockets for remote DB2 access from LUW. Will normal HTTP cut it? What type of throughput are you expecting when compared to JDBC?

Short answer. I see no restrictions using db2sock as a REST JSON interface. Web sockets, is simply another 'wrapper' (same as socket, etc.).

Correct. XMLSERVICE is not a speed maniac. In fact, XMLSERVICE was designed 100% around flexibility. Secondary goal XMLSERVICE was extreme simplicity used in any language toolkit (proven). Essentially, we can almost run an entire IBM i machine using XML to XMLSERVICE (shell, cmd, pgm, srvpgm, db2, etc.). Aka, we accomplished the goal with XMLSERVICE, which never was 'supreme performance' (*).

Today, db2sock, we are talking about attempting to keep the 'extreme flexibility' of XMLSERVICE, switch to JSON (arbitrary to performance), but really speed things up with some trade-offs. In fact, this is what we are all about with the SQL400Json new semi-architecture API.

To point, you guys are involved to accept the outcome of this experiment. Aka, look upon the result and judge 'good enough' for all things i want to do. This is the true meaning of Open Source in my view.

Optional read ....

(*) A bit of personal whining for XMLSERVICE author. In many ways, i was disappointed hearing demands of ridiculous performance expectations out of 'web workloads' (sarcasm). I recall sombody downloading 30 thousand records to a spread sheet (XML), looped thousands of times, 1 CPU machines with 1000 hits a second expectations, so on. XMLSERVICE was never intended to handle that sort of abuse. In fact, i have always made clear a custom DB2 stored procedure should be used when ultra high speed performance was required. Perfect knowledge between client and server in a stored procedure is always best. Anyway ... learn from a mistake. In this db2sock case, we will try to get the expectations correct together (you guys are the team).

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

I have probably made your head hurt. Sorry. For fun, relative to 'not limited narrative' let's pile on another wild 'heretic' idea to think about ...

The argument ...

DB2 performance may not be most important deciding factor in a web application. I know. Just saying such a thing may lead to hysteria (IBM i people fainting in the isle, etc.).

Consider ...

The relative 'code path' performance 'distance' of a request from browser to target server is 'nearly infinite'. Replace 'browser' with 'REST API' (scripting language request), 'distance' remains same 'nearly infinite'. All in all a 'web request' takes three days in a covered wagon to travel across the internet to ask for a bit of data from a database ( relative computer time imagined ).

So?

Well, consider traditional scripting language 'database connection' attempting to run as fast as inhumanly possible to feed up three rows of JSON data. But, but, but, data returns by browser/rest covered wagon appropriately about three days work to requester (computer relative time).

What if?

What is the true perceived performance impact on a web application that used a 'pure json' database driver (PHP samples tests). That is, db2sock under a Apache RPG CGI with JSON REST requests instead of faster than speed of light traditional language database driver (like ibm_db2). Yes, of course, first answer is 'way too long'!!!! However, in fact, most rich client browser web page interfaces are actually multiple REST requests via Javascript (AJAX, JQuery, etc.). Aka, argument embodied, our internet already works with REST database requests all the time (80-90% of the time).

And ...

We already have a db2sock Apache RPG CGI module with persistent connections (right now). Not finished, but available.

The sharing ...

This original thought contemplated what it means to be a 'database request' on the web. Perhaps, sharing such a 'heretical' thought could lead to a new generation of thinking about what a script database driver should actually become. Aka, 80-90% of web database requests should actually be JSON over db2sock (Apache), not a traditional driver (ibm_db2). Therein, all the web convention simply work out-of-the-box (kerberos, basic auth, ldap, on and on).

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

you have to let your imagination run a bit to see all the possibilities, but there are many 'password-less' notions including for DRDA in IBM i

I actually did a DRDA presentation of this exact scenario.

You make mention of RPG-CGI and that's more pointedly what I am curious about; 2-tier with LUW. Today I have customers using things like the jdbc npm for DB2 for i access from CentOS/Ubuntu. The former XMLSERVICE would have been too slow for frequent database access. Would be great if this db2sock could do WebSockets for remote DB2 access from LUW. Will normal HTTP cut it? What type of throughput are you expecting when compared to JDBC?

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Oh, man, short answer 'there are no limits'.

always talking 1-tier with db2sock, correct?

No. Short version, let's say we pre-start a set of connections with full db,uid,pwd,id (4 items, last "id"), Now, later, by web convention allowed attached by only "id" (after start). Therein, say db="wisconsin" was non-LOCAL, we have a multiple tier application where db2 json/toolkit/data/sql is served per normal DRDA on another machine.

Mmm ... you have to let your imagination run a bit to see all the possibilities, but there are many 'password-less' notions including for DRDA in IBM i (i forget the pre-id/pwd set-up CL CMD that goes with WRKRDBDIRE). Again, whatever ILE can do, we can do with db2sock, because ILE can call the APIs like SQL400Json.

IBM i user profile who started the web server to do the authentication

Also no. There are IBM i specific notions like Apache %%client%%, EIM, LDAP, etc., that may switch profile in a ILE CGI based on a "id". The db2sock project already supports Apache RPG CGI 'json call' (binary ascii RPG), therefore any option available to Apache is also available to db2sock (right now). Again, basically anything you can do with a ILE program, can be 'inherited' by db2sock (again see include RPG CGI).

I realize this is an unsatisfactory answer. Aka, people from Missouri are reputed to require "see it to believe it". Due respect to fast NGIX, ILE Apache, ILE DRDA, ILE EIM, etc., have capabilities to do most anything.

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

The basic meaning ... 'i am running under web convention authentication kerberose, basic auth, etc., so i am able to "connect" with "id" token to do database work. Aka, essentially trust in web snake oil, wherein authentication mother Apache basic auth knows best, so you do not need a db2 db, uid, pwd.'

First, I am assuming we are always talking 1-tier with db2sock, correct? Second, in the above scenario is it using the IBM i user profile who started the web server to do the authentication?

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Sneak peek (some already works, but not all up in git project) ...

The following JSON gives general idea. Basically, as with all JSON, idea is to minimize text (json factoring).

Standard ...

Samples below follow a minimalist idea of positional parameters. Specifically, drop a parameter off the end and default is assumed. This is a standard optional parameter scheme "parm":[p1,p2,...].

Non-standard (hopefully) ...

However, when a parameter 'context' can be determined missing parameters that precede may be 'understood'. This is a non-standard idea. Example: "pconnect":["id"] missing db, uid, pwd, therefore we know single parameter means persistent connection "id". Of course, "id" only opens a security hole, so we may only allow such option under a pre-start env var (set CONNECT=ALLOW_ID). The basic meaning ... 'i am running under web convention authentication kerberose, basic auth, etc., so i am able to "connect" with "id" token to do database work. Aka, essentially trust in web snake oil, wherein authentication mother Apache basic auth knows best, so you do not need a db2 db, uid, pwd.'

#!c

/* json
 * request {
 * -- toolkit database --
 * "query":"select * from bobdata",
 *   "fetch":"*ALL",
 * "query":"call proc(?,?,?)",
 *   "parm":[1,2,"bob"],
 * "connect":["*LOCAL","UID","PWD"],
 *   "query":"call proc(?,?,?)",
 *   "parm":[1,2,"bob"],
 *   "fetch":"*ALL",
 * "pconnect":["id"],
 *   "query":"select * from davedata where name=? and level=? and reports=?",
 *   "parm":["bob",1,2],
 *   "fetch":"*ALL",
 * -- toolkit commmand --
 * "cmd":"ADDLIBLE LIB(DB2JSON)",
 * -- toolkit program --
 * "pgm":["NAME","LIB","procedure"],
 *   "dcl-ds":["name",dimension, "in|out|both|value|return", "dou-name"],
 *   "dcl-s":["name","type", value, dimension, "in|out|both|value|return"],
 * -- complex parm (example) --
 * "pgm":["CLIMATE","MYLIB","RegionTemps"],
 *   "dcl-ds":["regions_t",0,"in"],
 *     "dcl-s":["region","5a","TX"],
 *     "dcl-s":["region","5a","MN"],
 *     "dcl-s":["region","5a","", 20],
 *   "end-ds":"regions_t",
 * -- single parm --
 *   "dcl-s":["countout","10i0",0,"both"],
 * -- complex return value --
 *   "dcl-ds":["temp_t",999, "return","countout"],
 *     "dcl-s":["region","5a"],
 *     "dcl-s":["min","12p2"],
 *     "dcl-s":["max","12p2"],
 *   "end-ds":"temp_t",
 * }
 */

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

Anyway, please take care not to publicly call a companies baby 'ugly'. Aka, no vendor names or vendor products please. I really want us Open Source jockeys stay far clear of liable monkey business.

The Hibernate ORM is an open source project with implementation flavors, not a vendor. I will steer clear of naming vendors.

FWIW, I like it when expound on your views and document the "why". I've learned a lot in the process.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

PHP 7 ... great. Can i impose upon you to match my php version 7.1.2?

#!shell
bash-4.3$ php -v
PHP 7.1.2

(*) The only reason i ask for 7.1.2 ... php added 'version defines in headers' to prevent mismatch compiled pecl extensions (ibm_db2, etc). Yes. Argh! True pain for 'programmers', but understandable for production administrators. Case in point talk about a 'no back door' policy, pecl version thing qualifies (Aaron comment).

[kadler]

Original comment by Chris Hird (Bitbucket: ChrisHird, GitHub: ChrisHird).

---

Tony

I will work on getting a later version of PHP. I will need time as unable to install currently available Apache Server for IBM i with PHP version greater than mentioned. Need to think about how to do this (Dave Dressller did get PHP7 working on IBM i so may work out how to build my own env using it.) .

Chris...

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Chris, Per our JSON task, we need to find a common php version. I wish to provide a modified ibm_db2 driver. To end, I believe you mentioned your IBM i php was 5.4.0? I don't have anything that old to match php versions.

Can you find a more modern php? Perhaps php 7?

Again, i do not care which vendor. I suspect 'performance' will be a key topic in this JSON activity. PHP7 better performing version of the language (**).

(**) if only php 5.4 on IBM i available (yikes), i can try to get ibm_db2 to work on that expired version.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Mmm ... Aaron ... i don't recall if any company considers ORM to be the finest product/architecture ever (cat's meow, bee knees, 52 skidoo?). Anyway, please take care not to publicly call a companies baby 'ugly'. Aka, no vendor names or vendor products please. I really want us Open Source jockeys stay far clear of liable monkey business.Thanks for your support.