strictly typed hashes: allow to define hash structure in runtime

[pvanek]

Just a backlog, Related to #990

Use cases: sqlutil (db table structure read when it is required), soap structures etc...

[davidnich]

the ability to return a typed hash from (ex) a DB query would be great - consider the following code:

*list rv = sla_events.selectRows(sh, SqlDataOpt);
return map cast<hash<SlaPerformanceInfo>>($1), rv;

this has an O(n) performance penalty for imposing the hashdecl type on each element of the return value. If we could provide the return type to the selectRows() method, it would be more efficient.

The problem with dynamic runtime hashdecl generation is that the types aren't known at parse time but rather only at runtime.

Allowing a predefined type to be used for the return value of this method would be ideal - I can currently think of two ways of implementing that: 1) add a new Datasource* method and curreponding SqlUtil option that allows for an argument specifying the hashdecl type for the return value (ideally in conjunction with the new reflection implementation) - like

list<hash<SlaPerformanceInfo>> rv = sla_events.selectRows(sh, SqlDataOpt + ("return_type": "SlaPerformanceInfo"));

2) implement metaprogramming support (i.e. generics or templates) - like

template <typename T>
list<T> selectRows(...) {}

For SOAP use cases, the current code generation solution could generate hashdecls for the generated code to make it safe.

The idea of implementing C++-style const lvalue support would also solve a lot of problems regarding accessing invalid hash keys due to typos etc.

[pvanek]

note about soap and code generators - that's fine, but it is not universal. It cannot be used for tools like "soap ui" without hacks (run generator through system(), then load hashdecl into new Program() etc. Any change in WSDL requires re-generation of code.

I know it works fine for now, but it looks uncomfortable to me.

And this usecase can be valid for XML/DTD, json schemas, anything else

[davidnich]

@pvanek a solution where the interface description can change but no code changes are needed would only support trivial cases where the strong typing described here wouldn't matter anyway. I don't understand the comparison with SoapUI - SoapUI deals with messages and not code as far as I understand it.

[pvanek]

well, maybe I'm so weak with describing things ;)

The WSDL is an example here only. I'd like to have "strong typed" approach for hashes almost everywhere.

Let's assume I'm writing a (qorus) user interface for manual submitting SOAP messages (but it can be anything, swagger jsons, json schema validated uploads, anything). In nthis case it would be great to have nice UI which will do some validations for you.

So I have a WSDL, which is unknown in parse time. Let's say an user loads it into the UI, or he selects it from a list of available WSDLs. Then the idea is to take a description of WSDL and construct an UI based on metadata info of each element - name, data type, mandatory flag etc.

I think it can benefit from hashdecl implementation if there is something like "description of hashdecl" out of the box.

But maybe (and it's possible) I just don't understand the real need for hashdecl since the beginning ;)

[davidnich]

@pvanek I still don't understand what you mean by a strongly typed approach for hashes. If you mean that you would like to get a runtime error when you try to access an unknown key in a hash without first checking that it exists, then I would rather solve this with a const keyword that defines read-only properties to lvalues.

The idea with hashdecls is to define allow for strongly-typed hashes with validation at parse time and runtime. I understand it doesn't meet your requirement, but would such a const implementation instead?

[pvanek]

Sorry, I understood it that 'const' is about C implementation internally. So how the 'const' can be used? Like this?

const hash h = ( "foo" : 1, "bar" : now() );

h.foo = 2; # will it fail because the whole hash is 'const'? or will it be OK, because only the structure is 'const'?
h.xxx = 1 # will it fail because it's a new key?
h.foo = "1" # will it fail because it's string or would it be converted to int?

etc.

[davidnich]

@pvanek my idea was to implement const as a keyword for Qore making lvalues read-only. For example it could be useful in parameter declarations, meaning that the lvalue could not be modified in the function, method, or closure body.

A const lvalue would not be modifiable, and also we could implement logic where referencing an unknown key of a const hash would result in a runtime error.

For example with queries we could have:

const hash h = db.selectRow(sql);
# this could result in a runtime exception
if (h.typo_in_column_namee == 2) {
}

Note that in general I would like to avoid runtime errors though and push as much verification / validation to parse time (yet another idea from @tethal that I find very good).

[gamato]

I think those are different use cases. What Petr means, I believe, is that he wants an exception when reading a non-existing member of a hash. Right now we get NOTHING instead. The current behaviour is convenient sometimes, but very dangerous in many other cases. For instance, in a recent project we had many bugs when code did not match data (for various reasons) and such bugs could go unnoticed for long time (and I believe many are still hidden out there). I would prefer much stricter mode of operation for Qore (perhaps a new directive) that would raise exceptions instead of returning NOTHING (be it for hashes, lists, classes, whatnot). const does not solve this, it's a different use case.

[davidnich]

@gamato my suggestion was to implement const in such a way that it would behave exactly like this