Closed modelica-trac-importer closed 6 years ago
modelica-trac-importer
commented
6 years ago Comment by henrikt on 10 Nov 2015 10:02 UTC To discuss details of flattening, wouldn't a clear definition of what a flattened model is (in other words a definition of the intermediate format commonly referred to as Flat Modelica) be a reasonable prerequisite?
This is related to #1157.
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 10 Nov 2015 10:35 UTC Flat Modelica is more about syntax of a flattened model.
Yes, the informal definition of a flat model at the beginning of chapter 5.6 should replaced by a more formal one. Maybe you can provide such, which also fits to the output of the flattening process.
At the end we can write out the flattened model, either using a big part of the modelica syntax specification or only XML, and define it to be "flat Modelica". It only should contain all information.
modelica-trac-importer
commented
6 years ago Comment by henrikt on 10 Nov 2015 10:58 UTC If I recall correctly, the main obstacle for #1157 was not how to map the different intermediate representations to a specific format such as XML, but lack of definition of what flattened models etc really are. Defining Flat Modelica would have profound consequences and is not something that I or any other individual should do in isolation.
Irrespective of the underlying intention behind #1157, I would strongly prefer a primary definition of Flat Modelica as a source code format similar to Modelica, so that flattened models would be both human readable and human writable. I don't suggest to consider the mapping to XML for the purpose of this MCP.
modelica-trac-importer
commented
6 years ago Comment by jmattsson on 10 Nov 2015 11:42 UTC Is it really a requirement to use an algorithmic approach of describing it? My biggest problems with the current description is that the stepwise, semi-algoritmic nature makes it hard to understand what the result should be. I can't simply implement the steps described in our architecture, so I need to understand the result, rather than an example of how you could implement it.
It is possible that it will be clearer with a pure algorithmic description, but I want to make sure that we avoid a situation where the specification dictates the implementation rather than the result.
modelica-trac-importer
commented
6 years ago Comment by jmattsson on 10 Nov 2015 11:59 UTC After reading the documents (I apologize for making the first comment before that), I see that the proposed new text is very close to the actual implementation in JModelica.org. So for us it wouldn't cause any problems. I do still think that we need to keep in mind that we should avoid dictating implementation details.
I agree that the conceptual (or possibly actual) building of an instance tree makes it easier to describe actual effects of different language elements. I think that this can be a large improvement on the current text.
modelica-trac-importer
commented
6 years ago Comment by choeger on 10 Nov 2015 12:01 UTC (I will only post my criticism here to keep the comment short. Please do not understand that as a general rejection, I think everyone agrees on the importance of such a work)
The definitions are still too informal. There is neither concrete syntax for a class tree nor for an instance. Please consider providing concrete syntax for both, so we can exchange (human readable) examples and actually discuss the algorithm. There should be nothing in your head about this process that cannot be written down unambiguously (also: please no tree graphics etc. - make the whole thing textual)
The MCP ignores recent related work (http://www.ep.liu.se/ecp/118/068/ecp15118637.pdf) please consider this work.
The Proposal seems to lookup superclasses only in the class tree:
Classes of extends clauses of the current class are looked up in the class tree This seems to remove a feature from Modelica, since at least some tools allow for the inheritance of inherited classes. Please clarify.
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 10 Nov 2015 13:17 UTC I agree that we need to work on this, and this seems like the right direction and focus.
However, looking at the MCP it seems that it still has not resolved one of the basic problems of the existing specification - splitting the instantiation for each class into even more phases.
I also agree with J. Mattsson that a less algorithmic description is both desirable and needed (this is strongly related to the previous item; the reason is that in combination they will break all the strange problems with lookup "loops" for modifiers.)
(Oh, and I would recommend changing .dotx to .docx extension.)
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 10 Nov 2015 13:22 UTC Replying to [comment:6 choeger]:
- The Proposal seems to lookup superclasses only in the class tree:
Classes of extends clauses of the current class are looked up in the class tree This seems to remove a feature from Modelica, since at least some tools allow for the inheritance of inherited classes. Please clarify.
Even in the class tree it is possible to lookup classes through extends by looking up the class of extends, and then continuing at the found class. I will add this to the text.
modelica-trac-importer
commented
6 years ago Comment by msasena on 10 Nov 2015 14:08 UTC I think this MCP would be a great addition to the clarity of the spec. Thanks for those taking the lead! A few comments to consider:
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 10 Nov 2015 14:35 UTC Replying to [comment:7 hansolsson]:
However, looking at the MCP it seems that it still has not resolved one of the basic problems of the existing specification - splitting the instantiation for each class into even more phases.
The instantiation is practically a recursive procedure, where the three possible class element kinds (class, component, extends) are handled similarly (lookup class, apply redeclarations, create an instance, merge modifications and apply them, when at simple element). If these things are done, the same function is called again one the elements on level below. Transforming the recursion away would lead to a loop where the handling of elements is inside and a stack would keep track of the position in the tree. So there is no natural way to split the complete procedure vertically or horizontally in different stages.
On way could be to describe first the instantiation without redeclarations and modifications, focussing on class lookup. Next taking additionally redeclarations into account and describe their specialties, and finally describe the merging and application of modifications.
This would be a possible splitting (which takes the inner loop to the outside) and one could think about.
I also agree with J. Mattsson that a less algorithmic description is both desirable and needed (this is strongly related to the previous item; the reason is that in combination they will break all the strange problems with lookup "loops" for modifiers.)
Could you give an example of a lookup loop? Because of the statical existence of the complete class tree and the facts that
the later resolution of the names in the modifications in the flattening phase does not interfere with the instantiation process. The instance tree does not change and the names can be resolved without strange effects.
(Oh, and I would recommend changing .dotx to .docx extension.)
I dit it.
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 10 Nov 2015 14:42 UTC Replying to [comment:9 msasena]:
I think this MCP would be a great addition to the clarity of the spec. Thanks for those taking the lead! A few comments to consider:
...
- Can someone clarify why an algorithmic description how to flatten a Modelica model is a bad idea? I realize that tools may have their own techniques for handling certain aspects of the flattening, but the overall process itself should be uniform, shouldn't it?
As indicated above the problem with an algorithmic description is two-fold:
As an example consider: "In the first step, all necessary libraries including the model which has to be instantiated are loaded from e.g. file system and form a so called class tree."
This is algorithmic and even non-causal (we don't know the necessary libraries yet; that is ignored below - but adds a similar issue).
The algorithmic part here is:
Now assume that there is a syntax error in one of the mo-files of Modelica, but it is in an unrelated sub-package that is not needed.
The algorithmic description makes it clear that we have to load the classes, so one would expect the syntax error is reported - even when the class is not needed. In that way an algorithmic description may prevent optimizations, and to give tools the freedom to perform these optimizations a different description is needed. (The change is not as drastic as one would think.)
modelica-trac-importer
commented
6 years ago Comment by jmattsson on 10 Nov 2015 14:59 UTC Replying to [comment:11 hansolsson]:
In that way an algorithmic description may prevent optimizations, and to give tools the freedom to perform these optimizations a different description is needed. (The change is not as drastic as one would think.)
I think it is OK with an algorithmic description, if it is clear that it is the result that is required (not the process), that it is somewhat straightforward to figure out what result the algorithm gives for simple models, and that issues such as what happens in Hans' syntax error case are addressed.
However, I don't think we should have "algorithmic description" as a goal. The important things are that the description is more formalized and that is covers all the situations we have had discussions about. Since work has been done on this description, I suggest that we do not throw it aside just because it is algorithmic.
(For a syntax error in a file that is not actually needed by the model, there I'd actually suggest that it be up to the tool if it leads to a compilation error. While that might mean that some tools might reject the model and some accept it, all tools should still agree that there is an error somewhere in the library.)
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 10 Nov 2015 19:29 UTC Replying to [comment:11 hansolsson]:
As an example consider: "In the first step, all necessary libraries including the model which has to be instantiated are loaded from e.g. file system and form a so called class tree."
BTW: In the document this sentence is followed by the non-normative text: [It is a matter of implementation to avoid the loading of libraries at once at the beginning and to load them later on demand.] which indicates that this is not required, but it is conceptional, to simplify the description.
modelica-trac-importer
commented
6 years ago Comment by choeger on 10 Nov 2015 20:35 UTC Replying to [comment:9 msasena]:
- Can someone clarify why an algorithmic description how to flatten a Modelica model is a bad idea?
It is not. If we want anything, then we want two things: Our specification needs to be sound (i.e. a result should always be correct) and it should be deterministic (i.e. we should be able to deduce one and only one result for a given input). The only practical way to do so is by providing an algorithmic specification. A non-algorithmic specification lacks the second requirement (it might be possible to get multiple different correct results from e.g. a constraint-based specification). However, I do not see any problem with an algorithmic specification as long as it completely describes all prerequisites. The current MCP (as well as the spec) does not do so - e.g. it does not specify what a Class Tree is (in a complete, human readable manner) and uses terms like "reference" etc. that are highly implementation specific.
A classical approach would be to describe the input in the form of an inductive structure and the algorithm as a set of reduction rules or a transformation function. This is however, a feat that requires alot more than 2000 words.
Maybe it would be better to take a step back and decide first, how the "more formal" specification should actually look like?
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 11 Nov 2015 07:13 UTC Replying to [comment:14 choeger]:
A classical approach would be to describe the input in the form of an inductive structure and the algorithm as a set of reduction rules or a transformation function. This is however, a feat that requires alot more than 2000 words.
There could be an appendix of the specification containing the detailed formal definition and the deduction rules to describe the transformation function. If you would provide an initial version, where we can discuss about, it would be fine. In the specification itself I would prefer to have a real readable description, to help beginners and non specialists on informatics to understand, how Modelica works.
modelica-trac-importer
commented
6 years ago Modified by dietmarw on 11 Nov 2015 08:11 UTC
modelica-trac-importer
commented
6 years ago Modified by dietmarw on 11 Nov 2015 08:12 UTC
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 30 Nov 2015 15:02 UTC Martin O. proposed to setup a web meeting to get an agreement about the style of specification of this topic before the design meeting. Therefore I made a doodle poll to fix the date. If you are interested in please use this link: http://doodle.com/poll/wvsnwcvg27pn9755 The question is basically whether it is sufficient to have a better description using the current style of specification or a real formalized specification is necessary. Maybe there are also steps in-between.
Hans: Please be interested. ;-)
modelica-trac-importer
commented
6 years ago Comment by leo.gall on 30 Nov 2015 16:21 UTC Replying to [comment:9 msasena]:
- It sounds like there may be some overlap with the work Toivo, Hilding and others did on describing an XML format for the Modelica syntax of a flat modelica file. Some of us were hoping they could extend that work to include semantics, so perhaps this MCP can address those gaps. I remember them presenting their work at Design Meeting 82, but I couldn't find anything in the Design Meeting repo about it (maybe it's stored somewhere else).
The material can be found in a separate project directory: https://svn.modelica.org/projects/ModelicaXML/trunk/documents See also #1157.
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 1 Dec 2015 12:32 UTC According to the poll result, the web meeting takes place at December 11th from 15:00 to 16:00 MET. We will do it using WebEx. Hopefully this is OK for everyone. I will send invitations shortly before.
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 9 Dec 2015 11:09 UTC Replying to [comment:13 kurzbach]:
Replying to [comment:11 hansolsson]:
As an example consider: "In the first step, all necessary libraries including the model which has to be instantiated are loaded from e.g. file system and form a so called class tree."
BTW: In the document this sentence is followed by the non-normative text: [It is a matter of implementation to avoid the loading of libraries at once at the beginning and to load them later on demand.] which indicates that this is not required, but it is conceptional, to simplify the description.
I agree that this is good, but I believe this should be clearer, e.g. in 5.6.1 add: - An implementation may delay and/or omit building parts of these trees, which means that the different steps can be interleaved. If an error occurs in a part of the tree that is not used for the model to be instantiated the corresponding diagnostics can be omitted (or be given), and additionally if the error should only be reported in a simulation model it must be omitted since it is not used in the simulation model. - To me it is important that this is normative text to avoid confusion; and also that the implications for error messages is clear. Finally we might consider replacing the word 'step' by a near synonym.
(Will provide more comments later.)
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 9 Dec 2015 13:50 UTC As indicated in https://trac.modelica.org/Modelica/ticket/1458#comment:12 I believe we should split the instantiation into sub-steps.
This would not require major changes of the text. Additionally I don't understand how to perform lookup during instantiation in the 'class tree' - and believe it should be 'instance tree'. I understand that some of this is handled using the "Generation of flat equation system", but I don't think that is enough and would prefer to perform all lookup in the same tree.
I thus propose to handle that replace the first proposed part of 5.6.1.4 by (I have updated the document, but don't want to commit without discussion): - If the class or component is only partially instantiated the modifiers including redeclaration of the element itself take effect.
For local classes and components in the current class, instances are created and inserted into the current instance. Modifiers (including class redeclarations) are merged and associated with the instance and the element is partially instantiated. [The partially instantiated elements are used later for lookup during the generation of the flat equation system and are instantiated fully, if necessary, using the stored modification environment.]
Classes of extends clauses of the current class are looked up in the instance tree, modifiers (including redeclarations) are merged, the elements of these classes are instantiated using the new modification environment, and are inserted into the current instance.
For local and inherited component declarations, the partially instantiated components are instantiated - redeclarations of components take effect. -- The two major steps are "partial instantiation" (merging modifiers and inserting the name) and "modifiers take effect including redeclaration".
I found it necessary to handle components and classes more uniformly due to the next point. -- Additionally I don't understand how to perform lookup during instantiation in the 'class tree' - and believe it should be 'instance tree'. There are two reasons for this:
As an example consider the following:
model MyModel
protected
package A
extends B(redeclare package C=B.D);
end A;
package B
package D
type T=Real;
end D;
replaceable package C end C;
end B;
parameter A.C.T x=2;
end MyModel;The point is that in order to find 'A' we need to search locally in 'MyModel'; and to find that 'A.C.T' actually exists we need to instantiate MyModel.A.
By using the "instance tree" for lookup I also believe we need to remove the "Inner/outer attributes of classes are ignored" in 5.6.1.3. This will allow 'outer package'; but it is trivial to check that it doesn't appear in most cases. -- We might improve the terminology, basically each class or component goes through the following steps:
The check that extends-clause-names are independent of inheritance is done by comparing result of step 4 and step 3. That's why we can either add local components in step 3 or 4 - it only matters if you want to use "extends Component.A;" which anyway is an error.
Step 1 is "partially instantiated" Step 2 doesn't need a name, since step 2 and 3 can be done together without interruption. Step 3 can possibly be left unnamed even if classes may be in this step during step 4. Step 4 is "instantiated for lookup"(?); and suffices to perform any simple lookup. Step 5 is "instantiated" (needed to actually simulate the model)
modelica-trac-importer
commented
6 years ago Comment by choeger on 9 Dec 2015 20:03 UTC Replying to [comment:22 hansolsson]:
I thus propose to handle that replace the first proposed part of 5.6.1.4 ...
I'd like to note on your proposal that you are using lots of "imperative" terms. I consider this one big problem with the current specification. After all, Modelica is a declarative language, yet we are talking about changes of the internal state of classes (without defining this state btw.). Don't get me wrong, this might be a good idea in an actual implementation, but in a specification I consider it expendable. The result of instantiating a class should be the same, regardless how often you do it. I consider it best practice to keep the spec as minimal as possible and thus not talk about any kind of change to a class.
My take on it would be to look at what we can do with a class:
Also, we need to figure out the class first, of course ;), so we need:
These steps are obviously mutually recursive (late binding) and that is where all these stages come into play (assuming we already found a class C):
Select "x":
Flatten "C":
Lookup "x":
This is, of course, just a sketch and all the stages Hans mentioned are in there, but I recon it much simpler to understand. Plus, it leaves lots of room for optimizations, so good opportunities for all the commercial vendors ;).
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 10 Dec 2015 08:38 UTC Comments for 5.6.2 (had planned to add earlier but the server didn't cooperate). The instance tree recursion seems unclear to me - it first seems to only discuss components (due to conditional declaration), and later includes non-components.
Additionally the resolve for some variables is not just a matter of simple type and record vs. non-record.
And the connect-handling also needs access to conditionally disabled variables.
Thus I propose the following: -- The instance tree is recursively walked through as follows:
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 10 Dec 2015 09:11 UTC Replying to [comment:23 choeger]:
Replying to [comment:22 hansolsson]:
I thus propose to handle that replace the first proposed part of 5.6.1.4 ...
I'd like to note on your proposal that you are using lots of "imperative" terms. I consider this one big problem with the current specification.
I understand this, and I agree that it seems desirable.
However, what I did was to try to modify the existing proposal - to make it clearer and to make the semantics closer to the current ones (ideally identical).
Rewriting it in a declarative way seems like a larger task, and:
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 11 Dec 2015 09:25 UTC eplying to [comment:22 hansolsson]:
Additionally I don't understand how to perform lookup during instantiation in the 'class tree' - and believe it should be 'instance tree'. There are two reasons for this:
- For a hierarchical name the static name lookup in 5.3.2 actually instantiates the classes - i.e. it uses the "instance tree".
- The first part of the static lookup is to search in the local class and inherited elements (and then proceed to enclosing scopes). The inherited elements rely on the instantiation - and we don't want to duplicate that.
As an example consider the following:
model MyModel protected package A extends B(redeclare package C=B.D); end A; package B package D type T=Real; end D; replaceable package C end C; end B; parameter A.C.T x=2; end MyModel;The point is that in order to find 'A' we need to search locally in 'MyModel'; and to find that 'A.C.T' actually exists we need to instantiate MyModel.A.
To find A.C.T it is not needed to instantiate MyModel.A. It can be done as follows:
Probably this procedure needs to be described as class tree lookup.
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 11 Dec 2015 09:44 UTC (continuation) Normally, to create the instance tree, it is not necessary to instantiate the classes themselves. The only case, where it is necessary is when inner/outer is used to specify the class of a declaration, e.g.:
model InstanceLookup
model A
inner package P
package Q
constant Real c=1;
end Q;
type T=Real;
end P;
B b;
end A;
model B
outer package P
end P;
package R = P.Q;
P.T v=R.c;
end B;
A a;
end InstanceLookup;
When such an outer reference is encountered the class lookup needs to be switched to the instance tree and the class has to be found there.
Btw. I think a model using this feature is not well defined, because its validity depends from the place where it is used.
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 11 Dec 2015 10:59 UTC Replying to [comment:26 kurzbach]:
To find A.C.T it is not needed to instantiate MyModel.A. It can be done as follows:
- find A locally
- try to find C locally in A -> not found
- go through the extends clauses in A -> find extends B -> find package B starting at A(recursively with the same algorithm)
- remember redeclare package C=B.D at B -> B(redeclare package C=B.D)
- continue to look for C in B(redeclare package C=B.D) -> find replaceable package C end C;
- its replaceable -> check whether there is a redeclaration of C -> find redeclare package C=B.D -> C finally found
- look for T in C=B.D -> its a short class definition -> find B.D starting at A (the scope of redeclare package C=B.D) (recursively with the same algorithm)
- look for T in B.D -> find type T=Real; -> done
Well, remembering the "redeclare" (and potentially merging them; especially considering constrainedby etc) is doing the same as the normal instantiation would do (at least the first steps of the instantiation).
I understand that a light-weight and optimized version of instantiation (similarly as described above) is needed for lookup in some cases, but to me that is an implementation issue - not something that must be in the specification.
I would even argue that it shouldn't be in the specification - since it will take time to do right, and also to maintain (since we would need to update the specification in two places).
We can instead focus on making the instantiation itself lightweight enough.
Probably this procedure needs to be described as class tree lookup.
Well, the current static name lookup in 5.3.2 is described as instantiating (or "flattening") the class. I prefer to keep that, and reuse the instantiation.
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 11 Dec 2015 15:12 UTC Replying to [comment:22 hansolsson]:
I thus propose to handle that replace the first proposed part of 5.6.1.4 by (I have updated the document, but don't want to commit without discussion):
After discussion I have now updated the spec-changes in:
https://svn.modelica.org/projects/MCP/MAinternal/MCP-0019_Flattening
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 8 Mar 2016 18:10 UTC Flattening group:
Discussion points:
Discussion about variable list for Flat equation system, and in general about Flat format. One point is that removing connect-equations should be done later. Another point is that the flat format should allow further simplifications in the same format.
modelica-trac-importer
commented
6 years ago Comment by choeger on 16 Jun 2016 09:38 UTC I've skimmed the latest proposal and it seems to be unclear about the interaction between lexical scoping and modifications. See #2013 for an example.
Please let us discuss the issue there and merge the conclusion here afterwards.
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 12 Sep 2016 16:22 UTC Discussion about new version (not yet committed): Scoping problem - scope is defined for class tree, so how to perform lookup in instance tree from current scope? Using corresponding class in instance tree for lookup.
Modification environment was unclear (see 7.2.2), and probably needs to be improved. How does modification environment develop during instantiation (including redeclaration)?
Would be good with graphics for examples - one idea was to start with svg in Inkscape and then convert to word.
An additional issue is that flattening, modifiers, and lookup are defined in different places in the specification; would be good to collect in one place.
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 14 Sep 2016 09:40 UTC Plan what to do for next steps:
modelica-trac-importer
commented
6 years ago Comment by perost on 27 Sep 2016 10:15 UTC Maybe this has already been discussed at the previous meeting, but I find the handling of extends a bit unclear in the current proposal:
Classes of extends clauses of the current class are looked up in the class instance tree, modifiers (including redeclarations) are merged, the elements of these classes are instantiated using the new modification environment, and are inserted into the current instance.
For local and inherited component declarations, the partially instantiated components are instantiated - redeclarations of components take effect.
The first issue I have is that in the first paragraph it's said that inherited elements are instantiated and then inserted into the current instance. But in the second paragraph it's implied that they are only partially instantiated, and fully instantiated in the next step. I'm not sure if it actually matters in which of these steps inherited components are instantiated, but right now it's a bit inconsistent I think.
Another issue is related to the generation of the flat equation system. Consider this model:
model A
Real x = y;
Real y = 1;
end A;
model B
extends A;
end B;The proposal quoted above says that x and y are inserted into B. It also says:
To resolve the names, a name lookup using the instance tree is performed, starting at the enclosing instance having the class of the scope.
The scope of the modifier "= y" on x is A, but what is the enclosing instance? The proposal only says that the elements of extended classes are instantiated, not the classes themselves. And even if there is an instance of A somewhere in the instance tree, looking up y in it would find the y in A, not the one inserted into B.
And if B were used as the enclosing instance instead we'd be able to find names not reachable from A. One could of course check that any name found was originally from A, but then there's the case where y is instead a package constant in an enclosing package. Maybe the improvements to the modification environment will resolve this?
modelica-trac-importer
commented
6 years ago Comment by choeger on 27 Sep 2016 10:42 UTC Replying to [comment:34 perost]:
The scope of the modifier "= y" on x is A, but what is the enclosing instance?
In my thesis, I have to solve the same issue (i.e. precisely define this distinction). My solution is a two-stage strategy:
y is found locally. This is memoized by replacing y with an access to a special name up(0).y (up(n) indicates that a free variable is found in the n-th enclosing class). (Since this analysis requires the evaluation of the superclasses, it is not precisely lexical, hence the quotes)B, up(0) is replaced by B (the outermost extending class). The modification environment is obtained by following the path back to the original declaration of y through the inheritance hierarchy.tl;dr: The scope of y is A, the enclosing class depends on the context (might be any extending class).
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 27 Sep 2016 11:54 UTC Replying to [comment:34 perost]: You are right. To get it work it could be rewritten like this:
"An extends clause is seen as an unnamed component and is instantiated in the same way in the current instance, resulting in an unnamed instance node."
The sentence about the identity check has to be slightly extended:
"At the end, the current instance is checked whether their children with the same name are identical and only the first one of them is kept. Thereby unnamed children are crossed recursively and their named children are incorporated in the test as well. [This is important for function arguments where the order matters.] It is an error if they are not identical."
During generation of the flat equation system the sentence from above can be extended:
"To resolve the names, a name lookup using the instance tree is performed, starting at the enclosing instance having the class of the scope. This could be also an unnamed instance in case the class of the scope is instantiated from an extends clause."
If the name is not found the lookup is continued in the lexical enclosing class of the scope, not in the enclosing instance.
modelica-trac-importer
commented
6 years ago Comment by perost on 27 Sep 2016 17:03 UTC Replying to [comment:36 kurzbach]: There's still the issue that when looking for e.g. y in my example, it would be found in the unnamed component of the extends clause instead of in B. But I guess this can be resolved by inserting references to the elements in the unnamed component into B instead of inserting the elements themselves. That way the inherited elements would be stored in the unnamed components, but could still be found when doing lookup in the extending class.
modelica-trac-importer
commented
6 years ago Comment by eshmoylova on 6 Oct 2016 20:15 UTC After reading through the proposal (with and without the changes) and through all the comments I find that I'm struggling with understanding the most basic concepts that are being used. I am not sure if I missed them or they were not clearly stated.
What is an "instance" tree and how is it different from a "class tree"? Is a copy of a class tree, like a component is an instance of a class with local modifications and redeclarations?
What is "instantiation"? I understand that it is a creation of an instance tree. But that comes back to my first question. From comment:28 it looks like the term instantiation was used in place of flattening.
What is "partial instantiation"? There was some indication that it is instantiation in which modifications were not resolved. But it comes down to the question of what instantiation is.
If I understand correctly the instantiation is a process that walks the class and processes its extends clauses, local classes, and local elements. How exactly it processes them is my question number 2. What is also unclear is the order in which the elements are processed. In the current specification, it says that first the names of declared local classes and components are found. There was no mentioning of something similar in the proposal, but from comment:26 it looks like a similar step is done. The order of "finding" components is important when we encounter elements with the same name. The current specification and the proposal say that if the elements are identical the first is kept. However, there are still some unclear issues about "identical" and what "first" means as reported in ticket #2015. So the order of finding the elements is relevant.
It might be easier to consider an example. So, consider the following.
package P
model BT
parameter Real m = -1;
Real x;
equation
x = m;
end BT;
model M
model A
replaceable model AT
Real x;
parameter Real m = -10;
equation
x = m;
end AT;
AT at;
end A;
model B
model BT
Real x;
parameter Real m = 0;
equation
x = m;
end BT;
BT bt;
end B;
extends A(redeclare model AT = BT);
extends B;
end M;
end P;If we flatten/simulate model P.M what should be the value of at.x (at.m)? I tested this model in Dymola and OpenModelica. In Dymola at.x = -1. In OpenModelica at.x = at.m = 0. Which result is correct and how do you explain it from the point of view of the proposal?
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 7 Oct 2016 06:38 UTC Replying to [comment:38 eshmoylova]:
- What is an "instance" tree and how is it different from a "class tree"? Is a copy of a class tree, like a component is an instance of a class with local modifications and redeclarations?
The class tree is basically something like an AST, covering all the syntactical and textual information from the Modelica sources. The instance tree is constructed for a single model, which is to be simulated, from its class definition. For all components of it, subtrees are constructed from their class definitions and all redeclarations and modifications are applied. The purpose of the current rewriting of the proposal is to remove the class tree and explain all steps only on the instance tree.
- What is "instantiation"? I understand that it is a creation of an instance tree. But that comes back to my first question. From comment:28 it looks like the term instantiation was used in place of flattening.
In the proposal flattening stands for the whole process which consists of two steps:
- What is "partial instantiation"? There was some indication that it is instantiation in which modifications were not resolved. But it comes down to the question of what instantiation is.
The term "partial instantiation" comes from the original specification. For me it is still unclear, what "partial" really means. How much from a class definition is instantiated to sufficiently fulfil the "partial" property? In the proposal, the right sides of modifications are resolved not before the second step "generation of flat equation system". So in your interpretation, the first step performs only "partial" instantiation.
- If I understand correctly the instantiation is a process that walks the class and processes its extends clauses, local classes, and local elements. How exactly it processes them is my question number 2. What is also unclear is the order in which the elements are processed. In the current specification, it says that first the names of declared local classes and components are found. There was no mentioning of something similar in the proposal, but from comment:26 it looks like a similar step is done. The order of "finding" components is important when we encounter elements with the same name. The current specification and the proposal say that if the elements are identical the first is kept. However, there are still some unclear issues about "identical" and what "first" means as reported in ticket #2015. So the order of finding the elements is relevant.
The "order" is always the order of declaration in the surrounding class definition. This should be kept during instantiation. The terms "finding" and "identical" from the current spec are also unclear to me. The assumption is probably, if different elements are identical, is does not really matter, which one is chosen (because they are identical). The only places, where it matters are where order matters, and these are in functions and records. Then the first one in order of declaration is used, e.g from the first base class. Therefore "indentical" apparently means indentical including all elements, element types, redeclarations and modifications recursively down, or in other words, they have to have an identical instance tree.
It might be easier to consider an example. So, consider the following.
package P model BT parameter Real m = -1; Real x; equation x = m; end BT; model M model A replaceable model AT Real x; parameter Real m = -10; equation x = m; end AT; AT at; end A; model B model BT Real x; parameter Real m = 0; equation x = m; end BT; BT bt; end B; extends A(redeclare model AT = BT); extends B; end M; end P;If we flatten/simulate model P.M what should be the value of at.x (at.m)? I tested this model in Dymola and OpenModelica. In Dymola at.x = -1. In OpenModelica at.x = at.m = 0. Which result is correct and how do you explain it from the point of view of the proposal?
Here the question is, where the BT in the redeclation of AT is resolved. The scope of BT, where the lookup starts is M (surrounding full class definition) and the current specification says (5.3.1):
"This lookup in each scope is performed as follows • Among declared named elements (class_definition and component_declaration) of the class (including elements inherited from base-classes)."
So the BT from the inherited class B has to be found, so the result should be at.x = 0; From the point of the proposal, the only difference is, that the lookup is started at the surrounding instance node with the class of the scope. In this example it is the anonymous toplevel instance of class P.M, we are instantiating currently. Then, the base classes are examined, and if the name is found, here in B, and B is not instantiated yet, it is instantiated on the fly (including all redeclarations and modifications of extends B). The lookup is continued in the instance of extends B. So the instance node of P.B.BT is found. BTW, it seems to be a good idea to create nameless instances from extends clauses as decribed in comment:36.
modelica-trac-importer
commented
6 years ago Comment by eshmoylova on 7 Oct 2016 14:09 UTC Thank you, Gerd. I think I understand some things a little better. Let me try to explain the process the way I understood it, and you can correct me where I got it wrong.
If we are given a model to simulate, e.g. P.M in my example, we need to flatten it. By flattening we mean creating a model with one set of equations that comes from all components and all extends clauses with all names resolved to their unique identifiers. The model defines all the unique identifiers with their dimensions and modifications fully resolved (or their modifications become part of the equations, it can be implementation specific; the important part is that everything is resolved). All identifiers are directly in the model, in one "flat" layer. Any functions that are used are also included and also have all elements resolved and directly in the function and the names in the algorithm are fully resolved to the names defined in the function or in the model(?). This last is probably also implementation specific: whether to allow functions have names that are defined in the model or to make them completely stand-alone, so that if any constants that are used in the function but are defined elsewhere are converted into inputs.
The flattening process as described in the proposal consists of two steps:
The instantiation takes the class definition of the model and walks through all elements in order of declaration to bring them into the instance tree. You say construct instance subtrees for them. I'm trying to avoid this term because I am still not fully grasping the concept.
The instance tree knows where it came from. In my example, P.M will have a pointer to P.
For each element we do the following.
Let's consider my example. For the instance tree P.M we first discover the class A (model). We create a node in P.M instance tree called A, it's parent is P.M. We get its definition. Do we immediately go building it's instance tree or do we stop here until we need more? Then we find the class B. Processed the same way as A. Next we find the "extends A" clause. We create an unnamed node. Do we build the instance tree for the class A with its redeclaration immediately? OK, that's as far as I can get without further clarifications.
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 7 Oct 2016 15:05 UTC Replying to [comment:40 eshmoylova]:
Do we build the instance tree for the class A with its redeclaration immediately? OK, that's as far as I can get without further clarifications.
That is an important question - and one thing that we need to clarify (I have tried a bit - and will hopefully work more on it next week, so that we can then have a web-meeting about this before the design meeting).
The simple answer is that we cannot build fully the instance tree immediately, since there might be cross-dependencies between classes.
Thus we immediately perform some actions so that we can use class A to instantiate other classes (and itself and sub-classes). These action should neither require a complete instance tree nor build one (yet). After that we could build the instance tree for A, but we could also delay it even further.
The "partial instantiation" is then a name for one of those intermediate steps.
The non-simple answer involves making this easy/easier to understand, and define exactly what is needed to make this work.
The rest can be ignored: (Note that it isn't that unique to Modelica; what is maybe new is that you need to have several partially instantiated classes "active" - so it is not just a two-pass procedure for each class. E.g. when compiling a Java/C++-class you first need a pass to handle declarations in the class, and then a pass to handle function definitions in the class - in order to allow mutually dependent member functions defined inline in the class definition. (Templates are messier.) But I don't think you need to interleave this with other classes (except for inter-procedural optimizations).)
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 10 Oct 2016 06:58 UTC Replying to [comment:40 eshmoylova]: During the instantiation the general rule is: be as lazy as you can be. Instantiate only the part of a class definition you really need. Doing to much will be hopelessy inefficient, especially with respect to Modelica.Fluid.
The input for the instantiation of a subtree is only the element to instantiate, the class definition of its type, and the current modification environment, containing all active merged modifications and redeclarations. So in fact (as long as the class tree is still present), to instantiate a local class, you need only to instantiate a node with the name of the class, holding a reference to its class definition and a copy of the current modification environment. Thats it. (If there is no class tree anymore, this information has to be stored somewhere else.) So it is cheap, and you don't have to worry about cross references. May be this is meant, if we say "partially" instantiated.
If you want to instantiate an extends clause or a component, you have to lookup the class of it. You may find a local partially instantiated class. Fine, because she holds a reference to its definition, so you know what to do, and also, she has a modification environment which should be merged with your current, having a resulting enviroment which should be applied to the current instance.
To be able to find also locally defined (and modified) types, you have to instantiate them first.
Then, to be able to find types declared in base classes, you have to instantiate the extends clauses. Because extends clauses do not depend from their surrounding instance, you can instantiate each extends clause independendly with the current modification environment as far as you need.
At last you can instantiate the remaining components in the same way as you dit with the extends clauses.
It might be happen, that you will need to find a class inside a local, only partially instantiated class, because its class definition or one of its base classes contains an element with the name you look for. And may be, if the name is in located in a base class, you have also to create nodes for the extends clauses recursively while merging the modifications from the enclosing instance. Then you have create node below the local class instance and instantiate it partially, inheriting and merging the modification environment from its parent instance.
Finding constants inside of classes work in the same way.
At the end, there are several stages of instantiation (every stage comprises its predecessor):
The question is: how to describe this in the specification in a concise and declarative way? Is it allowed to abtract from the performance issues? Should we mention the different stages of instantiation or is it enough to say "partially instantiated"?
Concerning functions: During the generation of flat equation system, referenced functions are instantiated as usual classes to be simulated. They and their components have then automatically global unique identifiers. Constants (no matter whether they are inside or outside of functions) can be handled as global independed instances with their own modification equation.
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 19 Oct 2016 14:20 UTC Replying to [comment:42 kurzbach]:
Replying to [comment:40 eshmoylova]: During the instantiation the general rule is: be as lazy as you can be. Instantiate only the part of a class definition you really need. Doing to much will be hopelessy inefficient, especially with respect to Modelica.Fluid.
..
Finding constants inside of classes work in the same way.
At the end, there are several stages of instantiation (every stage comprises its predecessor):
- only one instance node with modification environment
- a node with children of local types
- a node with children nodes for extends clauses
- a node with children nodes for declared constants
- a complete instantiated node, ready for generation of equations Atleast, and it could be more.
I have tried to think about this, and hope to have some better proposal that we could discuss at a web-meeting before the next Design meeting.
The issues I found were:
The names below are tentative - but I believe the boundaries are the appropriate ones.
After some thinking I believe the following are the ones that are important:
So, based on the class tree we have top-level classes (partially instantiated - with empty modification environment).
Another name for "partially instantiated allowing lookup inside" would be "with partially instantiated children".
We can then go from "partially instantiated" to "partially instantiated allowing lookup inside" using the steps in 5.6.1.4:
To generate equations we need:
modelica-trac-importer
commented
6 years ago Comment by eshmoylova on 19 Oct 2016 15:48 UTC After Gerd's clarifications and comments on my example I think I understand the proposal better. But the best way I can describe it is from the last step to the first. So if instead of trying to define and clarify the terms used in the proposal, such as instantiation, we forget about them for a moment I would describe the process of getting the equations (backwards) as follows.
The final step is to get the equations. The equations are defined in terms of components and also come from the classes defining the components. So what we need to accomplish the last step is to collect (know) all components that are within our simulation model with their modifications, and for each class defining the components we then need to collect their components with their modifications and equations. There is goes recursively. We do not need to resolve modifications until we need to use them in equations.
In order for us to collect all components with their defining classes, we need two things:
If we try to accomplish these two things for each component as we find it we may not be able to lookup the defining types and/or modifications successfully. Just like in my example, 'at' is declared to be of type 'AT' which is redeclared as 'BT' which we can find correctly only after we extended 'B'. So before we can start looking up defining types we need to know all classes that are defined in our simulation model with their modifications and scoping environments. (Modifications will have their own scoping environment.) We do not need to know anything about the classes other then their definitions, modifications, and scoping environments until we find a component that uses the type. So in other words, we do not need to look up the components, classes, or equations defined in them until we "use" them as a type of some component. At that time we need to take into account any modifications applied by the component, so for each component we will create a separate instance of the defining type before we start finding components and classes inside it.
So if the final step it to walk through the components and their defining classes resolving equations, the two steps before that must be:
(the third last step) Collect all classes and components defined in the simulation models with their definitions, modifications (and modifications scoping environments), and their scoping environments by finding the elements defined locally and by bringing the elements defined in the extends clauses. The order of how we collect them is the order of declaration, not the way it is defined in the existing Specification: first process local elements then the extends clauses.
(the second last step) Lookup a defining class for each component in the simulation model, create a local instance of the class, apply (merge) the modifications specified by the component to the instance and apply the previous and this steps to the instance. When we apply the modifications at this point we do not need to resolve them until we need them. So for example, if we redeclared a class 'A' we do not need to look them up until we get to a step of looking up the defining class of a component whose defining class is 'A'.
I think these are all the steps we need. So if we put them in the correct order we can write them as follows.
I split the final step into two for more accurate algorithmic description. This is my understanding of how the flattening should be done based on my understanding of the proposal. I probably made some assumptions that are not specified in the proposal or may even contradict them. So it may not be an accurate re-wording of the proposal. I appreciate all the comments. Especially, from the authors' of the proposal. :-)
modelica-trac-importer
commented
6 years ago Comment by eshmoylova on 19 Oct 2016 16:06 UTC I forgot one very important step. We need to add equations from the extends clauses to the equations of the class with their scoping environment. We can do this in two ways. Either we can rewrite step 1 as follows.
Or we can keep a node representing the instance of the extended class with its elements and equations, and step 4 will look as:
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 20 Oct 2016 12:51 UTC Replying to [comment:44 eshmoylova]:
I split the final step into two for more accurate algorithmic description. This is my understanding of how the flattening should be done based on my understanding of the proposal. I probably made some assumptions that are not specified in the proposal or may even contradict them. So it may not be an accurate re-wording of the proposal. I appreciate all the comments. Especially, from the authors' of the proposal. :-)
I agree that this is an important part of the flattening (most of this is in 5.6.2 in the proposal), but it also needs the detailed definition of instantiation (especially base-classes) to ensure correct lookup. Similarly in your list I would skip step 1 - and let that be defined by the instantiation.
modelica-trac-importer
commented
6 years ago Comment by eshmoylova on 20 Oct 2016 13:39 UTC Replying to [comment:46 hansolsson]:
Replying to [comment:44 eshmoylova]:
I split the final step into two for more accurate algorithmic description. This is my understanding of how the flattening should be done based on my understanding of the proposal. I probably made some assumptions that are not specified in the proposal or may even contradict them. So it may not be an accurate re-wording of the proposal. I appreciate all the comments. Especially, from the authors' of the proposal. :-)
I agree that this is an important part of the flattening (most of this is in 5.6.2 in the proposal), but it also needs the detailed definition of instantiation (especially base-classes) to ensure correct lookup. Similarly in your list I would skip step 1 - and let that be defined by the instantiation.
Does it mean that what I described in step 1 is essentially how you see instantiation? If not, what are the differences?
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 21 Oct 2016 15:46 UTC Replying to [comment:47 eshmoylova]:
Replying to [comment:46 hansolsson]:
Replying to [comment:44 eshmoylova]:
I split the final step into two for more accurate algorithmic description. This is my understanding of how the flattening should be done based on my understanding of the proposal. I probably made some assumptions that are not specified in the proposal or may even contradict them. So it may not be an accurate re-wording of the proposal. I appreciate all the comments. Especially, from the authors' of the proposal. :-)
I agree that this is an important part of the flattening (most of this is in 5.6.2 in the proposal), but it also needs the detailed definition of instantiation (especially base-classes) to ensure correct lookup. Similarly in your list I would skip step 1 - and let that be defined by the instantiation.
Does it mean that what I described in step 1 is essentially how you see instantiation? If not, what are the differences?
Yes, it corresponds to the instantiation (but some details are missing).
It also made me realize that the instantiation itself is probably not needed as a separate step - which could make the naming easier.
We need the "partial instantiation" to perform lookup (defined one-level-at-a-time). We also need the "flattened equation system" for the hybrid DAE than we then can simulate.
But instead of starting with recursively instantiating the model we could start traversing the components (the partial instantiation allows this), which will indirectly instantiate the model.
The advantage of removing "instantiation" as an important separate named step is that we could simplify the names along these lines:
partial instantiation->instantiation with partially instantiated children->children-instantiated instantiated->should not be used, but could call it "recursively instantiated" flat equation system->flat equation system
modelica-trac-importer
commented
6 years ago Comment by kurzbach on 22 Oct 2016 15:00 UTC I think the proposed for steps and this kind of simplification go in the right direction. They will shorten the proposal significantly. They seem to cover most important aspects, but are general enough to leave enough freedom for realization. They do not force a specific implementation.
modelica-trac-importer
commented
6 years ago Comment by hansolsson on 3 Nov 2016 10:49 UTC I tried to update the specification proposal on the svn-server with the ideas above, and also added a variant where I replaced "partially instantiated" with "instantiated" as discussed above in r9516.
Reported by kurzbach on 10 Nov 2015 08:34 UTC This MCP is about the improvement of chapter 5.6 "Flattening Process" of the specification.
In the current Modelica language specification (MLS 3.3 revision 1) the procedure how to get the right DAE system of equations from a given model is not well defined. There are several ambiguities and understanding problems, resulting from the very short and abstract description.
The problems and missunderstandings are documented in several bug tracker tickets. Here's a list of tickets that can be summarized as "instantiation is not sufficiently defined":
638, #641, #661, #709, #789, #791, #841, #862, #863, #1033, #1043, #1081, #1219, #1314, #1397, #1458, #1574, #1645, #1649, #1654, #1680, #1685, #1687, #1696, #1716, #1734, #1735, #1745, #1772,
Out of these most of the discussion is on #1397 and #1458, but all of these tickets (and possibly more) contain important questions that needs to be answered when specifying the instantiation.
Because flattening is the central procedure when translating Modelica code into equations and finally to a simulatable model, the drawbacks have the consequence that some models are not really portable, and the Modelica language cannot be said to be well standardized.
Therefore this MCP tries to specifiy the flattening more clearly in an algorithmic, stepwise approach, taking also the generation of equations into account.
Location of the documents: https://svn.modelica.org/projects/MCP/MAinternal/MCP-0019_Flattening
The current status is that the chapter 5.6 should be complete (except of inclusion of your comments), but there are a number of chapters still need changes to be consistent with this new description, especially parts of chapter 7 should probably merged into chapter 5.
Please use this ticket for discussion and comments. This topic is important and urgent, so it should be discussed also on the next design meeting in Munich.
The current members of the working group are:
Thanks to Peter Fritzson who gave the initial push to make it.
Migrated-From: https://trac.modelica.org/Modelica/ticket/1829