List and Dict are Classes rather than DataTypes

[eclipse-qvt-oml-bot]

| --- | --- | | Bugzilla Link | 420150 | | Status | NEW | | Importance | P3 normal | | Reported | Oct 23, 2013 05:41 EDT | | Modified | Nov 04, 2015 04:20 EDT | | Reporter | Ed Willink |

Description

Forking off from Bug 419958.

Intuitively List and Dict are objects, so if you pass them to a mapping/query/helper as an inout parameter, the object in the caller may be updated by the call.

This is the behaviour of a Class Instance not a DataType Value.

Therefore ListType and DictType should extend Class rather than CollectionType.

Problem: If List is no longer a DataType, "=" will return false for two "equals" lists.

Option 1: Two Lists are always "<>" but may be "equals". (Just like Java)

Option 2: OCL allows the definition of "=" to be extensible/overloaded, so that Two lists may be "=". Corrolary: "=" cannot be used as a building block. Consider a Set<List> in which one of the lists mutates to be equal to another by value. Which of the List objects is ejected from the Set? Is it re-instated if it changes again?

Conclusion: Option 2 is not viable. Although OCL 2.3.1 11.6.2 Set says only "The Set is the mathematical set. It contains elements without duplicates." Surely "="/"<>" must form the basis of Set element discrimination? Therefore two Lists that are equal by value are not "=".

Problem: aList.oclIsKindOf(Collection) will return false. Too bad the conformance was a bit dubious anyway.

Minor change: List and Dict could/should then be constructed with "new". ListLiteralExp and DictLiteralExp get subsumed by IntantiationExp.

Since Eclipse QVTo already offers the intuitive, rather than the consistent behaviour, the bulk of this fix is to the specification and the metamodels.

Changing List::"=" and Dict::"=" may need a compatibility option and warnings.

[eclipse-qvt-oml-bot]

By Adolfo Sanchez-Barbudo Herrera on Oct 23, 2013 13:21

(In reply to comment #0)

Forking off from Bug 419958.

Intuitively List and Dict are objects,

To me, intuitively, a List is what a Sequence is. The only difference is that the former is mutable.

As we have been discussing, DataType doesn't imply immutability of their instances.

So, in principle, if Sequence is a DataType I see no reason why a List should be different

so if you pass them to a mapping/query/helper as an inout parameter, the object in the caller may be updated by the call.

This indeed might be a problem :. It's not a problem for prohibiting datatypes as imperative operation inout/out parameters, it's a problem of backwards compability for being allowing to do so... Saying that inout/out data types are not allowed, is not acceptable, IMO.

Note, that we have a similar concern with out parameters (not currently supported by QVTo implementation, though), which also affect OCL collection types:

helper compute(out aSeq1: Sequence(String), out aSeq2: Sequence(String)) : Boolean { ...\ aSeq1 := Sequence{"123"}\ aSeq2 := Sequence{"456"} \ }

However, QVTo Spec:

"All in or inout non null values, except for the primitive types are passed by reference", which sounds ideal since mutable lists and dictionaries are not primitive types.

What does "passing a List value by reference" mean?\ a) It makes no sense, that phrase is crap.\ b) the final list value of the VarParameter when the operation finishes, gets copied out to the variable which was used by the caller to pass the initial value in.

ImperativeOperation and ImperativeCallExp can perfectly deal with these semantics as it's specific to the semantic of an inout/out parameter, which sounds as a plausible alternative. It should no be a bad idea clarifying this across the specification, specially what respects to ImperativeOperation and ImperativeCallExp concepts.

Thoughts ?

[What it is really sound is having a DictionaryType inheriting CollectionType. To me, it should directly inherit DataType]

[eclipse-qvt-oml-bot]

By Ed Willink on Oct 23, 2013 14:50

(In reply to Adolfo Sanchez-Barbudo Herrera from comment #1)

(In reply to comment #0)

Forking off from Bug 419958.

Intuitively List and Dict are objects, To me, intuitively, a List is what a Sequence is.

Yes, hence the pragmatic design, which unfortunately does not work.

As we have been discussing, DataType doesn't imply immutability of their instances.

That is an unuavoidable consequence of their unreferenceability; you cannot tell whether the information is changed becuase the container has changed ciontent, or the container is different.\

so if you pass them to a mapping/query/helper as an inout parameter, the object in the caller may be updated by the call.

This indeed might be a problem :.

..\

However, QVTo Spec:

"All in or inout non null values, except for the primitive types are passed by reference", which sounds ideal since mutable lists and dictionaries are not primitive types.

What does "passing a List value by reference" mean? a) It makes no sense, that phrase is crap. b) the final list value of the VarParameter when the operation finishes, gets copied out to the variable which was used by the caller to pass the initial value in.

It's not crap. It is illegal. A DataType cannot be referenced so cannot be passed by reference. It can only be passed by value, which of course prohibits return.

If List and Dict are Classes (containing e.g a Sequence or a Map) the problem vanishes. Alternatively you could introduce OclDataType

to enable the value of a DT to be passed by reference. (OclDataType

could also support polymorphic passing of Collections if Collection-conforms-to-OclAny is retracted.)

[eclipse-qvt-oml-bot]

By Adolfo Sanchez-Barbudo Herrera on Oct 24, 2013 04:52

(In reply to comment #2)\

It's not crap. It is illegal. A DataType cannot be referenced so cannot be passed by reference. It can only be passed by value, which of course prohibits return.

Some refreshing lecture:

http://en.wikipedia.org/wiki/Evaluation_strategy\ http://en.wikipedia.org/wiki/Output_parameter#Output_parameters

Note that what I've proposed, is not new:

"in PL/SQL input parameters (IN) are passed by reference, and output and input/output parameters (OUT and IN OUT) are by default passed by value and the result copied back"

In the end, this is an specification language and CAN do whatever we want to do as long as it's clear, unambiguous, sensible and consistent. In practice, we shouldn't worry about if we are passing inout data types values by value or reference, since we are using Java to implement the specification and everything are objects, so the "result is copied back" is simulated thanks to Java characteristics without any trick. For out parameters, indeed, we could require some additional trick (and that's why they are not current supported... ?)

Your proposal, not only unnecessarily/unintuitively makes the divergence of lists from collections, but also I can't see how your proposal deal with out collections (or any other data type, different to a primitive type) parameters.

On the other hand, an interesting specification example, which may lead to misinterpretations:

helper Package::computeCandidates(inout list:List) : List {\ if (self.nothingToAdd()) return list;\ list.append(self.retrieveCandidates());\ return list;\ }\ }

If the inout parameter is meant to modify the input list, doesn't seem suspicious having to return it ?... Might be not, but it's not the best of the examples.

More thoughts. Why do we need out/inout parameters when the language allows you to return OrderedTuples to return multiple results ?... Perhaps, in mappings since you only want as result the elements to be traced. Besides, getting rid of in/out/inout/ parameters doesn't seem acceptable, too much breaking change.

In any case, we need to clarify how this evaluation strategy works for imperative operations.

Regards,\ Adolfo.

[eclipse-qvt-oml-bot]

By Ed Willink on Oct 24, 2013 10:41

(In reply to Adolfo Sanchez-Barbudo Herrera from comment #3)

In the end, this is an specification language and CAN do whatever we want to do as long as it's clear, unambiguous, sensible and consistent.

I don't really understand this becuase what is "this"? UML/OCL/QVTo/PL?

But in QVTo we certainly cannot do whatever we like.

a) QVT claims to be an extension of OCL, therefore cannot contradict anything in OCL.

b) OCL claims to be aligned with UML, so cannot contradict anything in UML.

So since UML has distinct Class and DataType, with DataType not referenceable and so not sensibly mutable we need to accommodate this.

Your proposal, not only unnecessarily/unintuitively makes the divergence of lists from collections, but also I can't see how your proposal deal with out collections (or any other data type, different to a primitive type) parameters.

I was getting carried away by thoughts on resolving Collection-conforms-to-OclAny. It's a bit strange, but I see no real problem with QVTo defining that an inout DataType parameter involves two non-obvious copies to propagate the internal result value back through the (in)out parameter to the source variable.

WFR. (in)out parameter must have an assignable 'source' (e.g. a Variable such as myList, but not an expression such as myList++).

More thoughts. Why do we need out/inout parameters when the language allows you to return OrderedTuples to return multiple results ?...

Tuples are needed to support 'inout' UML parameters in OCL. Not a problem as above for QVTo.

[eclipse-qvt-oml-bot]

By Adolfo Sanchez-Barbudo Herrera on Oct 25, 2013 04:16

(In reply to comment #4)

(In reply to Adolfo Sanchez-Barbudo Herrera from comment #3)

In the end, this is an specification language and CAN do whatever we want to do as long as it's clear, unambiguous, sensible and consistent.

I don't really understand this becuase what is "this"? UML/OCL/QVTo/PL?

I meant QVTo.

But in QVTo we certainly cannot do whatever we like.

a) QVT claims to be an extension of OCL, therefore cannot contradict anything in OCL.

b) OCL claims to be aligned with UML, so cannot contradict anything in UML.

So since UML has distinct Class and DataType, with DataType not referenceable and so not sensibly mutable we need to accommodate this.

Absolutely.

Your proposal, not only unnecessarily/unintuitively makes the divergence of lists from collections, but also I can't see how your proposal deal with out collections (or any other data type, different to a primitive type) parameters.

I was getting carried away by thoughts on resolving Collection-conforms-to-OclAny. It's a bit strange, but I see no real problem with QVTo defining that an inout DataType parameter involves two non-obvious copies to propagate the internal result value back through the (in)out parameter to the source variable.

I agree that it's not usual, as we are used to more modern languages which doesn't behave in this way, but also in/out/inout parameters seems to be a legacy thinking which has touched QVT language.

WFR. (in)out parameter must have an assignable 'source' (e.g. a Variable such as myList, but not an expression such as myList++).

I hadn't thought about this. It definitely makes a lot of sense, which, from my point of view, should only be applicable to DataTypes, because if we also consider Classes it seems to be a non desirable limitation. As QVTo is statically typed, this distinction shouldn't be a problem for the compiler and the evaluator.

More thoughts. Why do we need out/inout parameters when the language allows you to return OrderedTuples to return multiple results ?...

Tuples are needed to support 'inout' UML parameters in OCL. Not a problem as above for QVTo.

Could you depict an example about this ? I'm not sure I follow.

Regards,\ Adolfo.

[eclipse-qvt-oml-bot]

By Ed Willink on Oct 25, 2013 05:19

(In reply to Adolfo Sanchez-Barbudo Herrera from comment #5)

Tuples are needed to support 'inout' UML parameters in OCL. Not a problem as above for QVTo.

Could you depict an example about this ? I'm not sure I follow.

http://www.omg.org/issues/ocl2-rtf.open.html#Issue18255

[eclipse-qvt-oml-bot]

By Ed Willink on Sep 24, 2015 12:22

The later of this discussion is sensible; the original title and comment is stupid.

List and Dict are clearly defined as CollectionType and so DataType. They are consistent with value semantics. It is just unusual from an OCL perspective to have mutable CollectionTypes. But UML DataTypes are not immutable. It might be helpful to introduce a CollectionType::isMutable() method that QVTo can override.

The OCL surprise is an implementation pragmatism; same address is equal value avoiding the need for deep clones and deep value comparison. CollectionType::isMutable() would allow an implementation to adjust.

List and Dict are inconsistent with Class semantics. 'Distinct' Lists can be equal. We therefore just need to sort out the pass-by-reference problems and value semantics inconsistencies. I think we can preserve the existing pragmatics semantics by defining "inout T : aDataType" as a 'short form' for

..."in T : aDataType-in, out T : aDataType-out"...\ var T : aDataType := aDataType-in; .....\ aDataType-out = aDataType;

Specification now consistent. How an implementation actually realizes the 'short form' is an implementation detail.

(In reply to Ed Willink from comment #4)

So since UML has distinct Class and DataType, with DataType not referenceable and so not sensibly mutable we need to accommodate this.

No. Referenceable and mutable are separate. "4+1" is a mutation of "4" to give "5". No need for any references.

"my.age++" is a read-modify-write. It does not reference data types.

[eclipse-qvt-oml-bot]

By Ed Willink on Sep 25, 2015 13:29

pass-by variable accommodates the impossible Datatype reference.

Suggested resolution:

---

In 8.2.1.15 MappingOperation Executing a mapping operation replace:

All out parameters, including result parameters, have their value initialized to null. All in or inout non null values, except for the primitive types are passed by reference. However it is not legal to change the value when an object is declared with in direction kind.

by:

All Class parameters are passed or returned by reference. An in Class parameter may not be modified.

in DataType parameters are passed by value and may be modified by the mapping without affecting the caller.

result DataType parameters are returned by value.

inout and out DataType parameters are passed by variable, that is the caller passes a variable whose value may be updated zero or more times by the mapping. These updates are visible to the caller.

out and result parameters are initialized to null which must be a legal value for the parameter.

out and inout parameters may be assigned zero or more times by the mapping. The initial or last assigned value is visible to the caller.

in 8.2.1.12 Helper replace:

A helper is an operation that performs a computation on one or more source objects and provides a result. The body of a helper is an ordered list of expressions that is executed in sequence. When more than one result is declared in the signature of the helper operation, the invocation of the operation returns a tuple.

Unless the isQuery property is true, a helper may have side-effects on the parameters; for instance, a list may be passed and changed within the body and its effect is visible after the operation call termination. However it is illegal to create or update object instances within a helper operation except for pre-defined types like sets, tuples, and for intermediate properties.

by

Helpers and queries are operations that perform a computation on one or more source objects and provide a result. The body is an ordered list of expressions that are executed in sequence.

A query has no side-effects. The isQuery property is true. All parameters are implicitly in.

A helper may have side-effects; for instance, a list may be passed and changed within the body and its effect is visible after the operation call termination. The isQuery property is false. Parameters may be in, inout or out. A helper may modify but not create Class instances. A helper may modify mutable DataType values such as List or Dict.

A helper or query may create mutable DataType values such as List or Dict or immutable DataType values such as Tuple, Set or String.

When more than one result is declared in the signature of a helper operation, the invocation of the operation returns a tuple.

All Class parameters are passed or returned by reference. An in Class parameter may not be modified.

in DataType parameters are passed by value and may be modified by a helper without affecting the caller.

result DataType parameters are returned by value.

inout and out DataType parameters are passed by variable, that is the caller passes a variable whose value may be updated zero or more times by the helper. These updates are visible to the caller.

out and result parameters are initialized to null which must be a legal value for the parameter.

out and inout parameters may be assigned zero or more times by the helper. The initial or last assigned value is visible to the caller.

[eclipse-qvt-oml-bot]

By Christopher Gerking on Oct 09, 2015 07:03

I may be wrong, but I recall that what you refer to as call-by-variable is actually named call-by-reference. The caller passes a reference to a variable. The value bound to that variable may be changed, whereas changes are visible to the caller. Java doesn't use call-by-reference.

Java uses a call-by-object-sharing, which allows object manipulation but assignments to the variable (if legal) are not visible to the caller.

I wonder if a separation between the call semantics and instance mutability would simplify the specification:

An 'out' parameter uses call-by-reference, no matter whether class or datatype. In order to be effective, it requires the caller to specify a variable or field that may not only be manipulated, but also reassigned by the call.

An 'inout' parameter uses typical Java call-by-object-sharing. Instance manipulation is possible. Reassignment may be possible but must not be visible to the caller. For immutable data types, this implies call-by-value semantics implicitly. The crucial question is whether these sharing semantics are allowed for datatypes at all. You seem to disagree, as you propose call-by-reference semantics for 'inout' data types. Why exactly? This seems to imply that we can't manipulate an existing List by passing it to an operation. Where's the difference between an 'inout' List and an 'out' List then?

An 'in' parameter uses call-by-object sharing as well. It can't use call-by-value because that would imply copying. For example, we might want to invoke 'resolve' on an 'in' class parameter which requires exactly the same object and no copy. To ensure the 'in' semantics, any mutation must be statically prohibited. Reassignment may be possible but must not be visible to the caller. For immutable data types, this implies call-by-value semantics implicitly.

[eclipse-qvt-oml-bot]

By Christopher Gerking on Nov 03, 2015 04:28

(In reply to Ed Willink from mail)

Yes call-by-reference-to-variable is a form of call-by-reference, but the reference must be to a varaiable that hosts the DataType value.

When you say call-by-reference, I assume that you are referring to what Java calls call-by-reference, right? Mutations to the given value are visible to the caller, but re-assignments are not.

There is no resolve() on DataTypes.

Why not? Datatypes can act as a mapping parameter, and there must be an implicit resolve in order to prevent a mapping re-execution either way. Eclipse QVTo supports resolve on datatypes as well.

I'm happy that the resolution is correct here, but perhaps the wording can be better.

How correct is it really? The Eclipse QVTo handling differs significantly, so please let me reflect from that viewpoint:

(In reply to Ed Willink from comment #8)

All Class parameters are passed or returned by reference. An in Class parameter may not be modified.

I disagree. An out class parameter requires "more" semantics than Java's call-by-reference. To use your words, it requires call-by-reference-to-variable in order to update the caller's variable binding.

in DataType parameters are passed by value and may be modified by the mapping without affecting the caller.

Fine. That's different from Eclipse QVTo, but I agree that local modifications are possible.

result DataType parameters are returned by value.

Agreed.\

inout and out DataType parameters are passed by variable, that is the caller passes a variable whose value may be updated zero or more times by the mapping. These updates are visible to the caller.

I still don't see the difference between inout and out for datatype params. Updates to the caller's variable binding should be visible only in case of out. Therefore inout requires different semantics, IMHO call-by-reference. For an inout List l, invoking l.add(...) should update the List passed by the caller. However l := List{} should not affect the caller at all.

out and result parameters are initialized to null which must be a legal value for the parameter.

Using the prevailing variable binding as the initial value is not an option?\

out and inout parameters may be assigned zero or more times by the mapping. The initial or last assigned value is visible to the caller.

Again I disagree with respect to inout. Assignments should not be visible to the caller, only mutations should be.

[eclipse-qvt-oml-bot]

By Ed Willink on Nov 04, 2015 03:22

Reviewing my words in the resolution (ignore my earlier words here), I see that my use of assign/modify/update lacks consistency and so you may easily be confused. Therefore, before going into the subtleties of List wrt Classes and DataTypes, please consider the following hopefully consistent wording using only assign/modify.

All Class parameters are passed or returned by reference.

An in Class parameter may be assigned without affecting the caller. An in Class parameter may not be modified.

in DataType parameters are passed by value and may be assigned or modified without affecting the caller.

result DataType parameters are returned by value.

inout and out DataType parameters are passed by reference-to-variable, that is the caller passes a reference to a variable. The parameter is an alias for the passed variable; assignments to the parameter, or modifications of the parameter's value apply to the variable and so are seen by the caller.

The initial value of each out and result parameter is null.

out, inout and result parameters may be assigned or modified many times.

The final value of each out, inout and result parameter is returned to the caller.

[eclipse-qvt-oml-bot]

By Christopher Gerking on Nov 04, 2015 04:20

Let's first clarify the basic meaning of in/inout/out. From the Eclipse QVTo perspective:

• 'inout' is the default which implies standard Java semantics: modifications are possible and visible to the caller, assignments (regardless of whether possible or not) are not visible to the caller.

• 'in' is a restriction of inout which prevents modifications.

• 'out' is an extension of inout which makes assignments visible to the caller.

Would you agree at least for class parameters?

(In reply to Ed Willink from comment #11)

All Class parameters are passed or returned by reference.

An in Class parameter may be assigned without affecting the caller. An in Class parameter may not be modified.

inout and out DataType parameters are passed by reference-to-variable, that is the caller passes a reference to a variable. The parameter is an alias for the passed variable; assignments to the parameter, or modifications of the parameter's value apply to the variable and so are seen by the caller.

The initial value of each out and result parameter is null.

So the only difference between out/inout parameters is that the initial value for out is null? I disagree.\

out, inout and result parameters may be assigned or modified many times.

The final value of each out, inout and result parameter is returned to the caller.

True for out. But IMHO assignments to inout parameters are not returned to the caller. The result must not be assigned after storing the trace record.