Indication of language version in file or class definitions

[modelica-trac-importer]

Reported by pharman on 20 Nov 2011 20:31 UTC There is no means of identifying from a Modelica file or class definition, what version of the language it is written for.

(To preempt the inevitable answer of the annotation(uses(Modelica(version="3.2"))) annotation, this only specifies library dependencies, so if the MSL is not a dependency there is no indication of version.)

I would like to propose adding a specification annotation to section 17.7.2, alongside version, conversion and uses, so that an annotation of the following form is possible: annotation(version="3.3",specification="3.3"), with the proposed text below.

---

• specification = SPECIFICATION-VERSION-NUMBER Defines that classes within this top-level class are based on, and use language elements defined in, version SPECIFICATION-VERSION-NUMBER of the Modelica Specification.

---

Migrated-From: https://trac.modelica.org/Modelica/ticket/643

[modelica-trac-importer]

Comment by choeger on 21 Nov 2011 11:23 UTC There is a problem with that approach: The syntax you will use for the language spec annotation is part of the language spec (and thus subject to change). Although it might seem unlikely that someone might change Modelica 3 annotation syntax, it might happen. Also it might be possible in the future to create hybrid projects of Modelica 3/4 models (along well defined interfaces). If that happens, we need a more general distinction. The best (but obviously unusable) decision would probably be to use different filenames:

model1.32.mo
model2.31.mo
model3.33.mo
...

This would probably confuse Windows (TM) users (and put an additional burden on text editor users). So instead I propose to use a comment section:

/*modelica_specification=3.3*/

It is much more unlikely that we are ever going to change the comment syntax.

[modelica-trac-importer]

Comment by pharman on 21 Nov 2011 11:37 UTC I was planning on raising the Modelica 3/4 interoperability issue in a separate ticket, but I certainly think that it will be essential, large industrial users are unlikely to want to switch versions overnight. However for that I would propose that Modelica 4 adopts it's own file extension such as .mo4. This would allow Modelica 3 and Modelica 4 models to coexist in the same path.

From a programming point of view, I don't really see that scanning for a comment containing certain text is easier than scanning for a Modelica annotation.

[modelica-trac-importer]

Comment by choeger on 21 Nov 2011 12:32 UTC Replying to [comment:2 pharman]:

From a programming point of view, I don't really see that scanning for a comment containing certain text is easier than scanning for a Modelica annotation.

My comment was not about ease of implementation, but rather about future version compliance. The problem is that we do not know how syntax might change right now. Since comments are part of the lexer, such a pragma should move in that stage IMO.

Another example: If we introduce a new keyword in Modelica 3.4 (very likely IMO), this kind of annotation is needed to decide what grammar we want to parse (3.2, 3.3 or 3.x ...), but if you need a parser to decide that you will never resolve that issue correctly.

[modelica-trac-importer]

Comment by pharman on 21 Nov 2011 12:46 UTC My point on ease of implementation was for that reason though, if you put code into a lexer in order to discover what the version is, then if it is in a comment you have to scan through every comment to find text matching a pattern. If you are looking for a pattern in the tokens returning from the lexer it is a lot more efficient.

I'm making the proposal because we have a genuine requirement, have implemented it, and see it as a weakness of the language as it stands.

If we introduce a new keyword in Modelica 3.4 it won't change this. What might affect it is if we have a big change in syntax when we move to Modelica 4, but I would like it to be considered in the design of Modelica 4 that knowing what specification version the tool should be using is necessary.

[modelica-trac-importer]

Comment by adrpo on 21 Nov 2011 13:04 UTC

Well, if you want to keep .mo extension for Modelica 4, then you need a clear way to differentiate between files. Annotations might change syntax, comments can get deleted or ignored by tools.

Maybe a good way would be a file identification grammar that is specification independent and appears at the start of the file:

grammar(specification = "3.4");
within ;
package Modelica
 ...
end Modelica;

You could read the grammar identifier with a non changing between specification versions lexer/parser. That of course if Modelica 4 will not be XML already :)

Cheers, Adrian Pop/

[modelica-trac-importer]

Comment by choeger on 21 Nov 2011 13:13 UTC Replying to [comment:4 pharman]:

My point on ease of implementation was for that reason though, if you put code into a lexer in order to discover what the version is, then if it is in a comment you have to scan through every comment to find text matching a pattern. If you are looking for a pattern in the tokens returning from the lexer it is a lot more efficient.

This is not the case: If you make

/*modelica_version=...*/

A token on its own, which is easy, you have no additional complexity at all.

If we introduce a new keyword in Modelica 3.4 it won't change this.

Yes it would. Every new keyword is a legal identifier nowadays. Because of this, a future Modelica 3.4 parser will probably not recognize Modelica 3.3 and so on.

Well, if you want to keep .mo extension for Modelica 4, then you need a clear way to differentiate between files. Annotations might change syntax, comments can get deleted or ignored by tools.

What I am trying to show here is that the Modelica version token would be a comment in older versions of the language and a token on its own in the newer versions. Then of course this (optional) token would be the first token in the token stream. Thus you could get the specification version basically in constant time.

[modelica-trac-importer]

Comment by mtiller on 24 Nov 2011 09:21 UTC I tend to agree with Christoph. It strikes me as problematic that you need to agree on some aspect of the grammar in order to determine which version of the grammar to use when parsing the file.

In some ways, this is similar to how shell scripts are run in Unix. The first line is a special comment that then tells the OS what interpreter to invoke.

Christoph's point about older versions is well taken. You want a syntax that will be recognized by all previous versions of Modelica as a comment, but all new versions as a special token.

To keep things simple, I'd suggest using the single line comment syntax along with at least one additional character to make recognizing the token. For example:

//! Modelica-Version: 3.2

In theory, this kind of syntax could be extended to allow other trans-version properties to be allowed. The generic grammar could be:

//! IDENTIFIER: LITERAL

(I leave it to the lexer/parser people to define something more formal)

Of course, the specification would have to have some basic text added about "any lines at the start of a file that start with this special prefix must be preserved", etc.

[modelica-trac-importer]

Comment by pharman on 24 Nov 2011 09:40 UTC There is surely a danger though that we end up with yet another flexible way of specifying things that is similar to annotations. Before you know it there will be instructions to the compiler appearing in comments. Plus it looks too much like the start of a Unix shell script to me!

My proposal to have this as an annotation was only for 3.x, and because I didn't want to propose something that added a language element. In an ideal world I think Adrian's suggestion is the best. If we can add an element at the top of the file along with the within clause then it is clear to both the developer and the tool.

Separating the Modelica language being used from the version of the MSL also has the advantage that when developing new features, the files containing the experimental feature can be marked as such:

grammar(specification = "3.3.ExperimentalNewFeature");
within ;
package MyPackage
 ...
  annotation(uses(Modelica(version="3.3"));
end MyPackage;

[modelica-trac-importer]

Comment by sjoelund.se on 24 Nov 2011 09:55 UTC Replying to [comment:8 pharman]:

Plus it looks too much like the start of a Unix shell script to me!

It looks too little like a Unix shell script to me. Introduce #![^\n]*\n as a lexical token and let us write #!/usr/bin/env modelica --std=Modelica3.3 the way we all want to (this is the way Python code looks, and many Modelicans seem keen on that language).

I like Adrian's proposal too. It's a quite nice way of specifying the grammar and could probably be treated as its own lexical token if need be (if we change the lexer substantially in Modelica4 for example).

[modelica-trac-importer]

Comment by dietmarw on 24 Nov 2011 10:04 UTC Replying to [comment:7 mtiller]:

To keep things simple, I'd suggest using the single line comment syntax along with at least one additional character to make recognizing the token. For example:

//! IDENTIFIER: LITERAL

I really like this simple syntax and I don't see it as a drawback that it looks like the start of a unix shell script. For me it makes especially sense because it looks like a script identifier. After all it does tell the tool how the parse what is following.

The problem I see with something like:

grammar(specification = "3.3.ExperimentalNewFeature");

is that again it is not backward compatible since a tool would need to understand grammar where ass the "hash-bang" sorry "double-slash-bang" way will be simply ignored by tools which don't know about it.

[modelica-trac-importer]

Comment by choeger on 24 Nov 2011 11:07 UTC That was what I meant: if Someone uses grammar or specfication as identifier, this change would not be compatible. What would be compatible was:

//grammar(specification = "3.3.ExperimentalNewFeature");

(Just in case the double-slash-bang notation scares someone away).

Just to get the proposal's point are we now talking about switching to some older version of Modelica or are we talking about activating some new features? In the latter case one can introduce whatever statement in the first line:

ThisCodeUsesFeatureFoobar and I want to show you!!!
within ExperimentalPackage

Since with feature foobar activated, the code is not standard Modelica anyway.

But if we want to be able to carray around legacy code, we need to find a way to annotate that code that still allows legacy tools to parse it.

[modelica-trac-importer]

Comment by pharman on 24 Nov 2011 11:22 UTC Replying to [comment:11 choeger]:

That was what I meant: if Someone uses grammar or specfication as identifier, this change would not be compatible. What would be compatible was:

//grammar(specification = "3.3.ExperimentalNewFeature");

(Just in case the double-slash-bang notation scares someone away).

True.

Just to get the proposal's point are we now talking about switching to some older version of Modelica or are we talking about activating some new features? In the latter case one can introduce whatever statement in the first line:

ThisCodeUsesFeatureFoobar and I want to show you!!!
within ExperimentalPackage

Since with feature foobar activated, the code is not standard Modelica anyway.

No but the majority of new features use backwards-compatible syntax anyway (new builtin functions, new annotations etc), so you wouldn't change the grammar every time you want to try something out.

The original proposal was for something backwards-compatible (and optional) in Modelica 3.x, so that we can track what specification is used by libraries. If we put it in 3.3 then we can see if something is pre-3.3 or what version it is from then on. But I would like to see something better (that doesn't have to be backwards-compatible) for Modelica 4.

But if we want to be able to carray around legacy code, we need to find a way to annotate that code that still allows legacy tools to parse it.

Maybe someone should propose this as an annotation? No, wait...

[modelica-trac-importer]

Comment by choeger on 24 Nov 2011 11:54 UTC Replying to [comment:12 pharman]:

No but the majority of new features use backwards-compatible syntax anyway (new builtin functions, new annotations etc), so you wouldn't change the grammar every time you want to try something out.

But still, what shall a tool do with a model that contains future (thus unknown) features?

Maybe someone should propose this as an annotation? No, wait...

As I said: An annotation might be subject to change. If we all agree that this does not happen, and if we find a generic way to put that annotation into excatly once for each file we can use an annotation. But since this is probably not going to happen, a pragma is IMO the better choice. It is not that no one in the world has ever had this problem, and there is a pragmatic solutions out there, so why not adopt it?

[modelica-trac-importer]

Comment by pharman on 24 Nov 2011 13:57 UTC Replying to [comment:13 choeger]:

Replying to [comment:12 pharman]:

No but the majority of new features use backwards-compatible syntax anyway (new builtin functions, new annotations etc), so you wouldn't change the grammar every time you want to try something out.

But still, what shall a tool do with a model that contains future (thus unknown) features?

I'm not saying an older tool could load experimental features, I'm saying a new tool can enable features based on it.

Maybe someone should propose this as an annotation? No, wait...

As I said: An annotation might be subject to change. If we all agree that this does not happen, and if we find a generic way to put that annotation into excatly once for each file we can use an annotation. But since this is probably not going to happen, a pragma is IMO the better choice. It is not that no one in the world has ever had this problem, and there is a pragmatic solutions out there, so why not adopt it?

I'm not against a pragma, I'd rather one that looked more "Modelica-like" rather than shell-script-like, but I'm fine with the idea of a pragma. I proposed an annotation because I didn't think people would agree with anything else and it is the normal means of storing additional information. The annotation I proposed was to be equivalent to the uses annotation so it isn't something that doesn't already exist. The proposal works fine for us, so I'm not going to agree to arguments of "that won't work", but I'd be very happy to see the issue of language version resolved if there is general consensus on a way to do it.

[modelica-trac-importer]

Comment by hansolsson on 12 Dec 2011 09:34 UTC Meta-comment about change of 'type' for this ticket. As I see it the indication of language version in file is an enhancement suggestion and not a 'defect'. Separting these two make it the handling easier.

[modelica-trac-importer]

Comment by hansolsson on 28 Feb 2012 10:52 UTC I do not believe this information should be in the package or even inside the file.

Basically you want to be able to determine Modelica version before parsing the file (one could imagine switching to different parsers depending on this version). If we parse with the wrong parser we might gets lots of problem - and even if the comments we want to parse is still valid there could be issues if we e.g. add new variants of string/quoted identifiers/comment/....

There are those two variants outside of the package:

within [packagename] annotation(ModelicaVersion="3.3"); or similarly with grammar as proposed above. This is not backwards-compatible.

A new file, e.g. package.version (contents yet to be defined), similar to package.order. Having a new file package.version is completely backward compatible and we can radically change the Modelica-language (even storing it as an XML-file) and still keep it. However, it only works for libraries stored as directories. For a non-file based storage it has to be stored as other form of meta-data.

[modelica-trac-importer]

Comment by hansolsson on 28 Feb 2012 11:32 UTC Language group: Very uncommon that a library is not using Modelica Standard Library; or use a vendor-specific annotation if not using MSL.

The within-annotation would work in Modelica 3.3, but not be backward compatible. A minimum would be to allow annotations on within in Modelica 3.3.

No real consensus.

[modelica-trac-importer]

Modified by pharman on 15 Aug 2012 08:10 UTC

[modelica-trac-importer]

Comment by pharman on 9 Apr 2014 10:03 UTC Replying to [comment:17 hansolsson]:

Very uncommon that a library is not using Modelica Standard Library; or use a vendor-specific annotation if not using MSL.

Having reported this issue based on the idea of not using the MSL, I have now realised (thanks to the data generated by Impact at https://impact.modelica.org/impact_data.json) that it is an issue right now even using the MSL: Modelica_Synchronous uses language features from the 3.3 specification, but it depends on Modelica 3.2.1. Therefore even the dependency on the MSL no longer specifies the language version that is in use.

There is a lot of value in automating package management of Modelica libraries, but a tool cannot automatically select libraries by language version because the information is not available. I therefore propose that an optional specification annotation would resolve this.

[modelica-trac-importer]

Changelog removed by pharman on 9 Apr 2014 10:03 UTC

[modelica-trac-importer]

Comment by hansolsson on 17 Jun 2014 16:25 UTC Spec group:

Outside the scope of 3.3r1. Adding a new file with version information would be backward compatible, and a tool can add it even if not specified in 3.3r1.

[modelica-trac-importer]

Modified by otter on 8 Jun 2015 09:45 UTC

[modelica-trac-importer]

Comment by dietmarw on 11 Jun 2015 14:52 UTC This was discussed at milestone:Design86 and resulted in

1726 (MCP-0015: Language Version).