Adding keyword parameters to existing constructors or sorts

[jurgenvinju]

1. Currently we have keyword parameters for constructors working exactly as they work for functions. This is good and consistent.
2. Since keyword parameters eventually will replace all use cases of annotations on constructors, we have a missing feature: to be able to add "annotations/kwparams" to existing definitions.
3. This feature, as far as we know, has no direct interpretation for normal functions. It's only relevant for constructors.

This thread is to discuss this feature, especially with @Anastassija, @mahills, @PaulKlint, @tvdstorm, @msteindorfer and @DavyLandman, to assert the need for it and possible syntax and semantics for this feature.

the need

• keyword parameters already satisfy the need of ignorability/hiding of certain information that is attached to nodes.
• they do not yet satisfy the need for extensibility of this information. This extensibility is the same extensibility that is offered by modular attribute grammar systems: to be able to compute and store information extra information for existing data types to satisfy the next orthogonal feature that is based on this abstract information.
• The canonical example: adding types to a syntax tree.
• We currently use annotations for this right now, with concrete examples in the parse generator (to annotate with unique identifiers), the syntax highlighter (to dynamically influence highlighting of parts of the tree, in abstract syntax trees in M3 (to refer to the original source code, and to refer to declaration names and to store types, in extensible code models (in M3 each relation is an optional annotation).

If we get rid of annotations without adding open extensibility to keyword parameters, then the above use cases are broken and left wanting. So let's agree that we need this :-)

the syntax

We might elevate the following existing syntax to work differently:

data Exp (int newParameter = 0);

This is a declaration of a sort without mentioning any constructor and using common keyword parameters.

the semantics

The semantics of the above would be to add not only this common keyword parameter to the constructors declared in the current module or the current definition, but to all visible constructors for Exp.

• problem: there may live constructors at run-time which have been built in the context where the keyword parameter was not defined. If this data is queried for the parameter, then the run-time value can not provide the answer.
• solution: code generation/interpretation should default to the visible keyword parameters in scope in case a parameter is not present on the run-time value.

Thoughts?

[DavyLandman]

I agree that to replace annotations we do need to be able to add keyword parameters to a data type after it's declaration.

I like the proposed syntax.

Would there ever be a possibility to pattern match on the kwparams?

[jurgenvinju]

Pattern matching works already! And yes, also on the externally added pnes, because there should be a default in place.—

On Fri, Aug 1, 2014 at 7:18 PM, Davy Landman notifications@github.com wrote:

I agree that to replace annotations we do need to be able to add keyword parameters to a data type after it's declaration. I like the proposed syntax.

Would there ever be a possibility to pattern match on the kwparams?

Reply to this email directly or view it on GitHub: https://github.com/cwi-swat/rascal/issues/626#issuecomment-50910762

[tvdstorm]

+1 for the syntax. I think the solution to the nodes without the extra stuff is ok too.

[tvdstorm]

To further complicate/empower things, what about the following.

Assume:

module Exp
data Exp = lit(int) | add(Exp, Exp);
int eval(add(x, y)) = eval(x) + eval(y)
int eval(lit(n)) = n;

Then:

module Var 
extend Exp;
data Exp = let(str, Exp, Exp) | var(str);
Exp eval(map[str,int] env = ());
int eval(let(x, e1, e2, env) = eval(e2, env = env + (x: eval(e1));
int eval(var(x), env) = env[x];

Note line four. Not entirely consistent the the data Exp(...) variant, but in the same spirit for functions.

Because of extend the env should be automatically passed on in the eval cases of Exp. I think @DavyLandman also asked about such automatic propagation.

NB: I don't know how general this is for extensible interpreters, -- it may just work accidentally for env. For instance, using a proper store would require to also track it in the result of eval.

[tvdstorm]

@jurgenvinju Re the "default that is in scope": what happens in the rare cases the same kwparam is added in different modules, with different default values, whose constructed values travel across module boundaries? Would this be observable?

The goal should be: from the local perspective from a single module, it should be consistent. IOW it should not be observable that some value originating in the other module had a different default value. But as it is now we cannot observe whether a kwparam's value is default or not in the first place, so maybe this can't trigger weird behavior.

[jurgenvinju]

@tvdstorm The function thing is proposed and discussed in depth elsewhere by @PaulKlint and @Anastassija in the general keyword parameters discussion: #154. It is really separate from the current more scoped discussion on adding keyword parameters to data types because there is no implicit parameter passing going on here. BTW implicit parameter passing, when applied to constructors leads to another feature at constructor level: inherited attributes.

Let's leave this part of the discussion to the bigger one about implicit parameters for functions and focus on just extensible keyword parameters for constructors here.

[tvdstorm]

And! We then don't need dynamically scoped variables anymore. :-)

[jurgenvinju]

@tvdstorm: since keyword parameter values are set at instantiation time (if declared in the current scope), then yes all these differences are observable and that is right (in my mind).

Yes, at least right now this is consistent (since we don't have the check for "setness" yet) and a default is just a transparent way of always having some value set for the parameter. But still, if we add "setness" to keyword parameters, then I don't see any issues. "setness" is not depending on the actual value (it can be "set" yet equal to the default), so this is orthogonal to the current discussion.

[jurgenvinju]

@tvdstorm about dynamically scoped variables: yes that was what @Anastassija and @PaulKlint had in mind, but not for this discussion. This involves a lot more, including variable references etc.

[tvdstorm]

No: if setness ("givenness") is possible, you can observe whether a value originated in this scope or not. I think this breaks modularity.

[jurgenvinju]

@tvdstorm how could you observe whether it originated in this scope then? I don't see it. You can only observe if somebody set the variable in any programmatic way other than the default computation. You can not see whether module A or module B was responsible for this.

[tvdstorm]

@jurgenvinju @Anastassija @PaulKlint Apologies for diluting the discussion: I assume this is the place I should be: https://github.com/cwi-swat/rascal/issues/154 ?

@cwi-swat/jurgenvinju says yes. A question is whether the current extension is consistent with the future changes we had in mind for #154, so far I believe it is. BTW, some of #154 is already represented in the compiler/run-time prototype.

[tvdstorm]

Maybe I'm wrong. Can't come up with an example. Lexical scoping probably saves you.

But if you extend a module that adds the same kwparam with different default there's an issue.

[jurgenvinju]

@tvdstorm the issue is which default parameter takes precedence? I would say that the default value is governed by the execution context at creation time of the constructor.

If you extend a module with the same kwparameter then there is no reason to add it again and you should probably be forbidden to do this. This kind of overriding is reserved, in my mind, for another feature -the explicit override feature- which we have yet to add to the semantics of extend (just like rename still has to be added to extend).

[PaulKlint]

Since data declarations are global, having two different defaults for the same keyword parameter is inconsistent.

The current design of common keyword parameters in data declarations also has its limitations: it only adds the common keyword parameters to the constructors in the current data definition (and not to previous or future ones).

[jurgenvinju]

[I edited this to remove shortsighted response from me]

Re: @PaulKlint: I agree that it can seem inconsistent from a global perspective, but maybe it's not that bad. Since we do have the module scope, if keyword defaults have module scope that would be ok. Do you agree? In a way this would be the same semantics as (overloaded) functions which are defined over several modules have. I also think that a static warning or even an error is in order for such clashes.

For modules that extend each other we do need to resolve inconsistencies in a different way, because then the original declaration and the new one may end up in the same scope. For this we should have the override feature for all kinds of declarations, including keyword parameter defaults.

This proposal is about removing the current limitations on common keyword parameters to enable the extensibility we need. The current limitations lead to an obviously correct semantics, so the question is whether removing them generates any more problems besides the unavoidable non-global properties of constructors.