Modelica.Utilities.Files.exist doesn't work if considered as a pure function

[modelica-trac-importer]

Reported by fcasella on 22 Jan 2016 11:02 UTC The Modelica.Utilities.Files.exist is an impure function, because it returns a different result depending on the state of the file system.

However, lacking the impure specifier, OpenModelica considers it as a pure function and, as long as the string input is a constant literal, moves its evaluation to the initialization phase, yielding wrong results in general.

The proper fix to this problem is to mark the function as impure. However, this is not allowed in Modelica 3.2.

Can we agree on a common strategy for all tool vendors for MSL 3.2.2?

---

Migrated-From: https://trac.modelica.org/Modelica/ticket/1886

[modelica-trac-importer]

Comment by otter on 22 Jan 2016 11:53 UTC Replying to [ticket:1886 fcasella]:

The Modelica.Utilities.Files.exist is an impure function, because it returns a different result depending on the state of the file system.

However, lacking the impure specifier, OpenModelica considers it as a pure function and, as long as the string input is a constant literal, moves its evaluation to the initialization phase, yielding wrong results in general.

The proper fix to this problem is to mark the function as impure. However, this is not allowed in Modelica 3.2.

Can we agree on a common strategy for all tool vendors for MSL 3.2.2?

There are the following alternatives:

1. Use "impure" keyword in MSL 3.2.2 (this means that MSL 3.2.2 will be based on Modelica 3.2 and uses additionally the keyword "impure" of Modelica 3.3).
2. Use Modelica Association "vendor annotation", e.g., "annotation(!__ModelicaAssociation_impure=true)".
3. Rely on vendor heuristics (e.g. all function calls in a when-clause and during initialization are considered as impure).

I am strongly for (1) because this is the best solution for all players (tool vendors, library developers, end-users). The only tiny drawback is that the version numbering of MSL is then not fully correct, because MSL should be then named MSL 3.3.1. However, since only one keyword of Modelica 3.3 is used, it seems complete overkill use such a version number.

Tools vendors, please state your opinion (if possible soon)!

[modelica-trac-importer]

Comment by sjoelund.se on 22 Jan 2016 12:01 UTC I am OK with (1) and (2). OpenModelica supports the annotation __OpenModelica_Impure=true which does the same thing as the impure keyword (OMC also supports the keyword). Supporting __ModelicaAssociation_impure=true would not be problematic.

[modelica-trac-importer]

Comment by beutlich on 22 Jan 2016 14:34 UTC This is a duplicate of #1478.

And if I remember correctly from the Modelica Design Meeting in Velicy we were about discussing if allowing impure and getInstanceName() for the next MSL is possible.

[modelica-trac-importer]

Comment by hansolsson on 22 Jan 2016 14:41 UTC Replying to [comment:1 otter]:

Replying to [ticket:1886 fcasella]:

The Modelica.Utilities.Files.exist is an impure function, because it returns a different result depending on the state of the file system.

However, lacking the impure specifier, OpenModelica considers it as a pure function and, as long as the string input is a constant literal, moves its evaluation to the initialization phase, yielding wrong results in general.

If the function is called outside of a when-clause it is an error that should be detected - and using impure will help with that.

However, moving call from inside a when-clause to the initialization phase is an unsafe optimization. Don't do that.

The proper fix to this problem is to mark the function as impure. However, this is not allowed in Modelica 3.2.

Can we agree on a common strategy for all tool vendors for MSL 3.2.2?

There are the following alternatives:

1. Use "impure" keyword in MSL 3.2.2 (this means that MSL 3.2.2 will be based on Modelica 3.2 and uses additionally the keyword "impure" of Modelica 3.3).

I agree with using impure - but not with the description "based on Modelica 3.2 with few keywords from Modelica 3.3".

To me it is based on Modelica 3.3, but except for the impure keyword compatible with Modelica 3.2.

1. Use Modelica Association "vendor annotation", e.g., "annotation(!__ModelicaAssociation_impure=true)".

Dymola already supports annotation(__Dymola_pure=false); and supporting a different one seems odd.

1. Rely on vendor heuristics (e.g. all function calls in a when-clause and during initialization are considered as impure).

I am strongly for (1) because this is the best solution for all players (tool vendors, library developers, end-users). The only tiny drawback is that the version numbering of MSL is then not fully correct, because MSL should be then named MSL 3.3.1. However, since only one keyword of Modelica 3.3 is used, it seems complete overkill use such a version number.

I have no problem with it being called MSL 3.3.1 (or 3.3.0?) - even if the synchronous parts are not used.

[modelica-trac-importer]

Comment by anonymous on 22 Jan 2016 14:44 UTC Replying to [comment:1 otter]: SimulationX also supports the annotation __iti_Impure or just Impure or ___Impure (all case sensitive). It does not support the keyword (because it is from Modelica 3.3).

[modelica-trac-importer]

Comment by otter on 22 Jan 2016 15:51 UTC Although this is a similar discussion as in #1478, lets continue the discussion in this ticket.

From the discussion, it turns out that there are more options. I list all of them:

1. Use "impure" keyword and call the next MSL version 3.2.2
2. Use "impure" keyword and call the next MSL version 3.3.0
3. Use a Modelica Association annotation (!__ModelicaAssociation_impure=true) and call the next MSL version 3.2.2
4. Use the existing vendor annotations and call the next MSL version 3.2.2: annotation(!OpenModelica_Impure=true, __Dymola_pure=false, \iti_Impure=true).
5. Do nothing and rely on vendor heuristics (e.g. all function calls in a when-clause are considered as impure).

Please, be constructive: There cannot be a perfect solution because only Dymola supports 3.3 and all other tool vendors only 3.2 (and maybe working on 3.3). Calling the MSL version 3.3.0 seems to be strange for a tool that does not support Modelica 3.3. Please, keep also in mind that it is already late, and changing now 3.2.2 Beta.1 to 3.3.0 Beta.2 might give other problems. It might also be not much effort for tool vendors to support the "impure" keyword, even if they do not support it now.

If we do not get full agreement, I will introduce the vendor-specific annotations from tool vendors that ask me (so at least __OpenModelica_Impure=true will be added). This will be introduced for the following 35 functions (because it is better to have vendor-specific annotations in the MSL, as to have the risk to get a wrong or an inapropriate solution):

• function readTableData in Modelica.Blocks.Sources.CombiTimeTable, Modelica.Blocks.Tables.CombiTable1D/1Ds/2D
• Modelica.Math.Random.Utilities.automaticGlobalSeed
• Modelica.Math.Random.Utilities.impureRandom
• Modelica.Math.Random.Utilities.impureRandomInteger
• 12 functions in Modelica.Utilities.Files
• 8 functions in Modelica.Utilities.System
• 8 functions in Modelica.Utilities.Internal.FileSystem

O.k., tool vendors, please state again which options are acceptable for you.

[modelica-trac-importer]

Comment by beutlich on 22 Jan 2016 19:08 UTC There are many impure functions missing in this list, e.g. from Modelica.Utilities.Streams, Modelica.Utilities.Examples or ModelicaTest. I propose to collect them all before any change is performed.

[modelica-trac-importer]

Comment by sjoelund.se on 22 Jan 2016 19:14 UTC And then you have things like reading matrix dimensions (that new function?), that needs to be pure since it is used for array dimensions...

[modelica-trac-importer]

Comment by beutlich on 22 Jan 2016 19:36 UTC Replying to [comment:8 sjoelund.se]:

And then you have things like reading matrix dimensions (that new function?), that needs to be pure since it is used for array dimensions...

Marking readMatrixSizes as pure and readMatrix as impure means that the array data is allowed to change only if the dimensions are not touched.

[modelica-trac-importer]

Comment by sjoelund.se on 22 Jan 2016 19:47 UTC Replying to [comment:9 beutlich]:

Replying to [comment:8 sjoelund.se]:

And then you have things like reading matrix dimensions (that new function?), that needs to be pure since it is used for array dimensions... Marking readMatrixSizes as pure and readMatrix as impure means that the array data is allowed to change only if the dimensions are not touched.

Yes, that was my thought.

[modelica-trac-importer]

Comment by otter on 22 Jan 2016 21:35 UTC After re-thinking, the only realistic solution seems to be to introduce vendor annotations. Since SimulationX recognizes the __OpenModelica_Impure=true annotation, the following annotation needs to be added to impure functions (other vendors should report their needed annotations also):

annotation(__OpenModelica_Impure=true, __Dymola_pure=false)

According to the discussion above, these annotations should be added to:

• function readTableData in Modelica.Blocks.Sources.CombiTimeTable, Modelica.Blocks.Tables.CombiTable1D/1Ds/2D
• Modelica.Math.Random.Utilities.automaticGlobalSeed
• Modelica.Math.Random.Utilities.impureRandom
• Modelica.Math.Random.Utilities.impureRandomInteger
• Modelica.Utilities.Examples.readRealParameter
• Modelica.Utilities.Streams.readLine
• Modelica.Utilities.Streams.countLines
• Modelica.Utilities.Streams.readRealMatrix
• Modelica.Utilities.Streams.close
• All 12 functions in Modelica.Utilities.Files
• All 8 functions in Modelica.Utilities.System
• All 8 functions in Modelica.Utilities.Internal.FileSystem

If you think there are more impure functions, please name them explicitly (I did not find impure functions in ModelicaTest)

[modelica-trac-importer]

Comment by beutlich on 22 Jan 2016 23:15 UTC

• The functions readTableData in ModelicaTest.Tables.CombiTable2D.Test15/Test16/Test17 are also impure.
• What about Modelica.Utilities.Streams.writeRealMatrix? Since it accesses the file system it also should be impure.
• What about the protected functions within impure functions, e.g., copyDirectory/copyDirectoryContents within Modelica.Utilities.Files.copy, removeDirectory/removeDirectoryContents within Modelica.Utilities.Files.remove or existDirectory within Modelica.Utilities.Files.createDirectory.

[modelica-trac-importer]

Comment by otter on 23 Jan 2016 14:54 UTC Replying to [comment:12 beutlich]:

* The functions readTableData in ModelicaTest.Tables.CombiTable2D.Test15/Test16/Test17 are also impure.

o.k. I am wondering, whether it would be better to have only one public definition of readTableData (e.g. in Blocks.Tables.Internal) instead of defining it 7 times in a protected section. This is done in other cases as well, e.g., Blocks.Continuous.Internal, Utilities.Internal).

* What about Modelica.Utilities.Streams.writeRealMatrix? Since it accesses the file system it also should be impure.

"impure" is a quite fuzzy term and one needs to know the context. In Modelica 3.3 the "impure" keyword is defined as:

With the prefix keyword impure it is stated that a Modelica function is impure and it is only allowed to call such a function from within:

• another function marked with the prefixes impure or pure
• a when-equation, or
• a when-statement.

I guess for the impure vendor annotations something similar holds. Therefore, all functions that can be called also outside of a when-clause (especially in an equation section) should not be defined as "impure". All functions that "write" something to a window or a file, can be called at every time step during continuous-simulation, because they do not influence the integrator ("do not introduce a memory that the integrator should be aware off") and since a data object is given as input argument, sorting is also fine, because this data object must first be computed (e.g. at the current time instant) and only then the "write" function can be called.

For example, this is already used with the "print(..)" function, e.g., for debugging purposes. To summarize, I do not think that writeRealMatrix should be marked as impure (in the sense of Modelica).

In the view of the Modelica 3.3 impure definition, it seems also forbidden to use an impure function in a parameter declaration. I quickly checked with Dymola, and Dymola allows to use an impure function in a parameter declaration. Anyway, this indicates Modelica.Utilities.Examples.ReadRealMatrixFromFile should not be marked as impure (because the typical usage is to call it in a parameter declaration, see Modelica.Utilities.Examples.ReadRealMatrixFromFile).

* What about the protected functions within impure functions, e.g., copyDirectory/copyDirectoryContents within Modelica.Utilities.Files.copy, removeDirectory/removeDirectoryContents within Modelica.Utilities.Files.remove or existDirectory within Modelica.Utilities.Files.createDirectory.

When the "impure" keyword would be used, then an impure function can only be called within another function, if this function is marked with the prefixes impure or pure. For vendor annotations this is probably uncritical, but to be one the safe side, one should mark them as well as "impure".

[modelica-trac-importer]

Comment by fcasella on 23 Jan 2016 14:54 UTC I would remind that we are trying to keep the MSL free of vendor-specific annotations. Of course vendors can have their own tool-specific annotations, but this is a bit of a complication, it is much nicer if tools can support the MSL as it is (in particular for MSL developers!)

So, I would suggest to use __ModelicaAssociation_impure=true, which I guess can be accepted in the MSL, as it is vendor-neutral.

[modelica-trac-importer]

Comment by otter on 23 Jan 2016 15:05 UTC Replying to [comment:14 fcasella]:

I would remind that we are trying to keep the MSL free of vendor-specific annotations. Of course vendors can have their own tool-specific annotations, but this is a bit of a complication, it is much nicer if tools can support the MSL as it is (in particular for MSL developers!)

So, I would suggest to use __ModelicaAssociation_impure=true, which I guess can be accepted in the MSL, as it is vendor-neutral.

I completely agree that MSL should be free from vendor-specific annotations. However, with impure functions and the time frame (it is planned to release MSL on March 11), there are no good solutions available and one has to select the least "bad" choice. Introducing now an MA annotation (never done before, and equivalent to existing vendor annotations) then the MSL forces all Modelica tool vendors to support it, or get bad behavior for impure functions. Worse, we have already an agreed solution, "impure" keyword of Modelica 3.3, and it does not make sense for me to introduce now yet another, just intermediate, standardized solution. So, clearly, the least "bad" choice is to use existing vendor annotations (vendors do not have to change anything in their tool), and to remove these vendor annotations again for the 3.3.0 release of MSL (and then using the "impure" keyword).

[modelica-trac-importer]

Comment by otter on 24 Jan 2016 20:05 UTC Fixed in 9675059d9b73520eca614086ffe3412d3acb49ad:

Introduced impure vendor annotations (SimulationX recognizes !___Impure=true)

annotation (__OpenModelica_Impure=true, __Dymola_pure=false

for impure functions:

readTableData in 
    Modelica.Blocks.Sources.CombiTimeTable, 
    Modelica.Blocks.Tables.CombiTable1D/1Ds/2D
    ModelicaTest.Tables.CombiTable2D.Test15/Test16/Test17
Modelica.Math.Random.Utilities.automaticGlobalSeed
Modelica.Math.Random.Utilities.impureRandom
Modelica.Math.Random.Utilities.impureRandomInteger
Modelica.Utilities.Examples.readRealParameter
Modelica.Utilities.Streams.readLine
Modelica.Utilities.Streams.countLines
Modelica.Utilities.Streams.readRealMatrix
Modelica.Utilities.Streams.close
All 12 functions in Modelica.Utilities.Files and
   opyDirectory/copyDirectoryContents within copy, 
   removeDirectory/removeDirectoryContents within remove
   existDirectory within createDirectory. 
All 8 functions in Modelica.Utilities.System
All 8 functions in Modelica.Utilities.Internal.FileSystem 

"Check" on Modelica and ModelicaTest is successful with Dymola 2017 dev.4

[modelica-trac-importer]

Modified by otter on 24 Jan 2016 20:06 UTC

[modelica-trac-importer]

Changelog modified by otter on 24 Jan 2016 20:06 UTC Marked impure functions with vendor-specific annotations for impure functions

[modelica-trac-importer]

Comment by beutlich on 25 Jan 2016 11:17 UTC One wrongly introduced annotation was removed by 2d1cdca8fa565938234650fb464707e8cf684540.

[modelica-trac-importer]

Comment by beutlich on 25 Jan 2016 11:39 UTC Replying to [comment:18 beutlich]:

One wrongly introduced annotation was removed by 2d1cdca8fa565938234650fb464707e8cf684540.

There are far more occurrences where the impure annotation was added to the external function annotation by error.

[modelica-trac-importer]

Comment by jmattsson on 26 Jan 2016 16:08 UTC Replying to [comment:11 otter]:

After re-thinking, the only realistic solution seems to be to introduce vendor annotations. Since SimulationX recognizes the __OpenModelica_Impure=true annotation, the following annotation needs to be added to impure functions (other vendors should report their needed annotations also):

Sorry for the delay, we had to make some quick design decisions. Please also use __Modelon_Impure=true.

[modelica-trac-importer]

Comment by otter on 27 Jan 2016 11:27 UTC Replying to [comment:19 beutlich]:

Replying to [comment:18 beutlich]:

One wrongly introduced annotation was removed by 2d1cdca8fa565938234650fb464707e8cf684540. There are far more occurrences where the impure annotation was added to the external function annotation by error.

You are right, there was one wrongly introduced annotation. Thanks for correcting this. However, I do not yet see that there are other errors. Please, give details

[modelica-trac-importer]

Comment by otter on 27 Jan 2016 11:32 UTC Replying to [comment:21 otter]:

Replying to [comment:19 beutlich]:

Replying to [comment:18 beutlich]:

One wrongly introduced annotation was removed by 2d1cdca8fa565938234650fb464707e8cf684540. There are far more occurrences where the impure annotation was added to the external function annotation by error.

You are right, there was one wrongly introduced annotation. Thanks for correcting this. However, I do not yet see that there are other errors. Please, give details

O.k. found that 3 annotations in ModelicaTest.Tables have been wrongly placed. Corrected this in 36aaa9c0c63558ee4d47dbd9a80980ee38160cf8

[modelica-trac-importer]

Comment by otter on 27 Jan 2016 11:36 UTC Replying to [comment:20 jmattsson]:

Sorry for the delay, we had to make some quick design decisions. Please also use __Modelon_Impure=true.

Added __Modelon_Impure=true to the impure functions in b5b3ec547532a45bfb7dc04b425f3baeb528ac69

[modelica-trac-importer]

Comment by beutlich on 27 Jan 2016 11:38 UTC For example it is wrong for readTableData.

[modelica-trac-importer]

Modified by beutlich on 27 Jan 2016 14:29 UTC

[modelica-trac-importer]

Comment by beutlich on 28 Jan 2016 16:34 UTC otter Are you fixing the remaining ones such that a fixed beta can be released?

[modelica-trac-importer]

Comment by otter on 29 Jan 2016 07:12 UTC I made a check, and to my understanding all annotations are correct now.

[modelica-trac-importer]

Comment by beutlich on 29 Jan 2016 07:34 UTC Looks like we do not speak the same language?! The Impure annotation should not be added to the external annotation but as function annotation.

Index: Sources.mo
===================================================================
--- Sources.mo  (revision 9017)
+++ Sources.mo  (working copy)
@@ -2297,8 +2297,8 @@
       input Boolean verboseRead
         "= true: Print info message; = false: No info message";
       external"C" readSuccess = ModelicaStandardTables_CombiTimeTable_read(tableID, forceRead, verboseRead)
-        annotation (__OpenModelica_Impure=true, __Modelon_Impure=true, __Dymola_pure=false,
-                    Library={"ModelicaStandardTables", "ModelicaMatIO", "zlib"});
+        annotation(Library={"ModelicaStandardTables", "ModelicaMatIO", "zlib"});
+      annotation(__OpenModelica_Impure=true, __Modelon_Impure=true, __Dymola_pure=false);
     end readTableData;

     function getTableValue

[modelica-trac-importer]

Comment by otter on 29 Jan 2016 07:54 UTC Fixed in 8e503443669a65a7404e5e1a70f7d01507c794f6 (if you find such trivial errors, please correct yourself, that is faster. Otherwise, give precise information in which file the error is present).

[modelica-trac-importer]

Comment by beutlich on 29 Jan 2016 12:54 UTC OK, fixed by 95896032cff5e5cb0b980c07d0f80f9aa9fd94e0.

[modelica-trac-importer]

Comment by sjoelund.se on 12 Feb 2016 14:29 UTC Replying to [comment:13 otter]:

For example, this is already used with the "print(..)" function, e.g., for debugging purposes. To summarize, I do not think that writeRealMatrix should be marked as impure (in the sense of Modelica).

But if Modelica.Utilities.Streams.print is not marked impure, a tool is allowed to optimize it away (OpenModelica does since it's a pure function returning nothing). And the thing here is that this function is allowed to write to a file, so it may affect the solver state.

[modelica-trac-importer]

Comment by beutlich on 12 Feb 2016 14:37 UTC Especially for the _ModelicaRequirements library all functions that call print (e.g., initializeLogFile, printViolationsToLogFile, printViolationsToOutput etc.) needed to be marked as impure (_!__itiImpure in that case). So yes, print and writeRealMatrix should also be impure.

[modelica-trac-importer]

Comment by otter on 18 Feb 2016 14:15 UTC Since we do not have time to discuss the precise definition of "impure" and since it does not matter, I have marked "Streams.print" and "Streams.writeRealMatrix" as impure in a5ff2ab76c2564e10b7b0b4200449af7d8cfb1ad.

Note, it might be that the Modelica specification is not fully clear about the definition of an "impure" function. However, the approach from OpenModelica to optimize away a function that does not have return values is not useful. If this would be the intention of Modelica, the specification would state that every function must have at least one return value. Please, just remove this optimization in OpenModelica (if this optimization has an effect, the model is always modified in an unwanted way, so why have you introduced this optimization?).

If one is extremly strict, then potentially every external function call can influence the internal state of a simulator, and therefore every external function call should be treated as "impure". In that case, it would be simpler to state in the specification: Every external function is an impure function. However, this restricts the language and does not allow to utilize useful optimizations. We should be pragmatic here, and trust the function developer and if he/she states it is not impure, one can hope that the function does not influence the simulation state (if the he/she did not make an error).

[modelica-trac-importer]

Comment by sjoelund.se on 18 Feb 2016 14:22 UTC Replying to [comment:33 otter]:

the approach from OpenModelica to optimize away a function that does not have return values is not useful. If this would be the intention of Modelica, the specification would state that every function must have at least one return value. Please, just remove this optimization in OpenModelica (if this optimization has an effect, the model is always modified in an unwanted way, so why have you introduced this optimization?).

I disagree. It is very useful. And we also do not remove every function call that does not have a return value. We execute the function before it is removed. Triggering assertions or calling ModelicaError causes the function to not be fully optimized away.

The concept of impure functions was introduced just for stating which functions cannot be optimized away, so why should MSL not mark such functions as impure?

[modelica-trac-importer]

Comment by otter on 18 Feb 2016 15:00 UTC Let me state this differently: Can you give ANY useful example where it makes sense that a function call is optimized away if the function does not have a return argument?

[modelica-trac-importer]

Comment by beutlich on 18 Feb 2016 15:04 UTC Shouldn't be functions that call impure functions also be impure? I just give Modelica.Math.Matrices.Examples.solveLinearEquations as one example but there are plenty more of them.

MLS 3.2r2 12.3: "An impure function (that may return different values at different calls despite having the same input argument values), may be called from within an impure function, from within a when-equation or when-statement, and during initialization."

[modelica-trac-importer]

Comment by otter on 18 Feb 2016 15:36 UTC Replying to [comment:36 beutlich]:

Shouldn't be functions that call impure functions also be impure? I just give Modelica.Math.Matrices.Examples.solveLinearEquations as one example but there are plenty more of them.

MLS 3.2r2 12.3: "An impure function (that may return different values at different calls despite having the same input argument values), may be called from within an impure function, from within a when-equation or when-statement, and during initialization."

We can only improve the handling of impure functions in MSL but cannot provide a complete satisfactory solution with MSL 3.2.2 (but better as with MSL 3.2.1). A satisfactory solution requires that the "impure" prefix definition is refined, that the "impure" prefix is supported by all Modelica tools and that the "impure" prefix is utilized correctly in MSL, so nothing what we can do now. Therefore, at some point in time (and I would say, this is today, Feb. 18, 2016) we have to stop further discussing impure functions for MSL 3.2.2 because we anyway cannot provide a 100 % save solution. MSL 3.2.2 is still better in this respect as MSL 3.2.1.

Regarding Modelica.Math.Matrices.Examples.solveLinearEquations, I do not see any problem here: The "print" function is called, but due to the "impure" annotation, the "print" should not be optimized away by a tool. The "solveLinearEquation" can be called in any context, and can be fully treated as pure function.

I see the "impure" annotation for "print" and "writeMatrixToFile" as a temporary hack in order that MSL 3.2.2 works better in some tools. However, it would be a complete desaster to introduce the "impure" prefix for "print" later and then a tool would give an error if the function where "print" is used would be not marked as "impure" as well. We need to clearly state in the specification that opimizating away a function with no output arguments (that does not influence the simulator state) is an error.

Anyway, please do not open this ticke again (feel free to make a new ticket with a milestone > 3.2.2).

[modelica-trac-importer]

Comment by sjoelund.se on 18 Feb 2016 15:40 UTC Replying to [comment:37 otter]:

Regarding Modelica.Math.Matrices.Examples.solveLinearEquations, I do not see any problem here: The "print" function is called, but due to the "impure" annotation, the "print" should not be optimized away by a tool. The "solveLinearEquation" can be called in any context, and can be fully treated as pure function.

It can only call an impure print if it is itself marked pure or impure.

[modelica-trac-importer]

Comment by otter on 18 Feb 2016 15:51 UTC Replying to [comment:38 sjoelund.se]:

Replying to [comment:37 otter]:

Regarding Modelica.Math.Matrices.Examples.solveLinearEquations, I do not see any problem here: The "print" function is called, but due to the "impure" annotation, the "print" should not be optimized away by a tool. The "solveLinearEquation" can be called in any context, and can be fully treated as pure function.

It can only call an impure print if it is itself marked pure or impure.

We cannot introduce a "pure" annotation in all functions where the print(..) function is called. This is not manageable, and we cannot do this.

You have to decide what is least problematic for OpenModelica:

• Keep it as it is now (probably, the worst think that happens is that OpenModelica prints a warning in the function where the print-statement is used).
• We remove the impure annotations from "print" and OpenModelica is not changed (this means that some models will not get a print-out and the user will not be satisfied).
• We remove the impure annotation from "print" and OpenModelica is corrected so that functions are never optimized away that have no return argument.

[modelica-trac-importer]

Comment by sjoelund.se on 22 Feb 2016 13:12 UTC loadResource is also marked impure, but is used outside of when-statements/equations. It also does not require any vendor-specific annotations since ModelicaServices.ExternalReferences.loadResource can contain the annotations instead.

[modelica-trac-importer]

Comment by beutlich on 22 Feb 2016 13:34 UTC

1917 is a new ticket with problems of impure functions.

[modelica-trac-importer]

Comment by otter on 22 Feb 2016 16:48 UTC Replying to [comment:40 sjoelund.se]:

loadResource is also marked impure, but is used outside of when-statements/equations. It also does not require any vendor-specific annotations since ModelicaServices.ExternalReferences.loadResource can contain the annotations instead.

Fixed in a63a163d7b06a56463622724bb1f8d22b35a85a7 by removing the impure annotations from Modelica.Utilities.Files.loadResource

[modelica-trac-importer]

Modified by beutlich on 24 Feb 2016 14:42 UTC

[modelica-trac-importer]

Comment by otter on 29 Feb 2016 15:43 UTC In 15beee17c8dff5a578b9a7f44e25f3a1b64d48e4 OpenModelica_impure annotation was removed from readRealMatrix with the argument: Some MSL example models uses these impure functions in parameter bindings, which OM does not allow for impure functions (only when-equation/statement and pure/impure functions).

I think the Modelon and the Dymola impure annotations should also be removed, because this looks unsymmetrical (some tools declare it as impure, others not). Furthermore, symbolic simplifications like common subexpression elemination are fine on this function (e.g. replacing two function calls with identical input arguments by one function call), because if the same matrix is read several times in one model evaluation, always exactly the same matrix is returned (under the assumption that the matrix on file does not change for one model evaluation).

The same holds for readRealParameter in Utilties.Examples.

[modelica-trac-importer]

Comment by beutlich on 29 Feb 2016 22:14 UTC Reopen for the very reason that there are still open questions on impure functions.

Also I'd like to rethink the decision if we really want to introduce vendor specific annotations for the first time in MSL. The option with the impure keyword was listed many times in this ticket but finally withdrawn.

Actually I'd prefer to have a vote within MAP-Lib if we should take the least "bad" choice or a proper design choice.

[modelica-trac-importer]

Comment by otter on 1 Mar 2016 07:40 UTC Replying to [comment:45 beutlich]:

Reopen for the very reason that there are still open questions on impure functions.

We cannot provide a fully satisfactory solution now. The current solution improves the handling of impure functions of MSL 3.2.1 and the few new impures functions of MSL 3.2.2.

Also I'd like to rethink the decision if we really want to introduce vendor specific annotations for the first time in MSL.

It is not for the first time. MA is not an academic organization but supports organizations to solve real-world problems (which means one has to weight pragmatism with 100% solutions).

The option with the impure keyword was listed many times in this ticket but finally withdrawn.

The reason is simple: "impure" is a Modelica 3.3 keyword.

• The previous discussion reveals that the design of 3.3 is not complete and most likely an additional description is needed. So, we would have to spend time on it and make test implementations. This just means that we can forget to release MSL 3.2.2 in the near future.
• All tool vendors have to support a Modelica 3.3 keyword, even if tools vendors are currently trying to support fully Modelica 3.2. It is not a clean strategy for a tool vendor to partially support 3.2 and partially support 3.3.
• If the "impure" keyword would be used, the MSL release would have to be renamed to 3.3.1. However, many documentation parts and tool vendor tests rely on the release number 3.2.2 now. We cannot change the release number shortly before the release candidate.

The only realisitic other option is to remove all impure vendor annotations from MSL and force tool vendors to introduce a heuristic for external C-functions. However, the tool vendors should comment on it, whether they could live with this option.

Actually I'd prefer to have a vote within MAP-Lib if we should take the least "bad" choice or a proper design choice.

If you want to make a vote, open a separate ticket and describe precisly what you want to vote on. Once Toni is back from vacation, he can take care off it.

[modelica-trac-importer]

Comment by hansolsson on 1 Mar 2016 09:03 UTC Replying to [comment:46 otter]:

The only realisitic other option is to remove all impure vendor annotations from MSL and force tool vendors to introduce a heuristic for external C-functions. However, the tool vendors should comment on it, whether they could live with this option.

We already manage that in Dymola (as far as I understand) and I don't see that why we would stop supporting that.

The reason I wrote "As far as I understand" is that I haven't seen the model causing this problem - and whether it is a modeling issue or tool issue as indicated in comment 4.

[modelica-trac-importer]

Comment by hansolsson on 1 Mar 2016 09:12 UTC Replying to [comment:47 hansolsson]:

Replying to [comment:46 otter]:

The only realisitic other option is to remove all impure vendor annotations from MSL and force tool vendors to introduce a heuristic for external C-functions. However, the tool vendors should comment on it, whether they could live with this option.

We already manage that in Dymola (as far as I understand) and I don't see that why we would stop supporting that.

The reason I wrote "As far as I understand" is that I haven't seen the model causing this problem - and whether it is a modeling issue or tool issue as indicated in comment 4.

Looking at https://trac.openmodelica.org/OpenModelica/attachment/ticket/3645/TestDelete.mo I can say that the model seems correct.

However, the optimization of assuming that functions are pure doesn't seem like valid for Modelica 3.2 since impure functions may be called in when-clauses, and there is no way of declaring them pure or impure.

12.3 Pure Functions Exception: An impure function is either an impure external function or a Modelica function calling an impure function. An impure function (that may return different values at different calls despite having the same input argument values), may be called from within an impure function, from within a when-equation or when-statement, and during initialization. -- (I already indicated this in comment 4.)

[modelica-trac-importer]

Comment by sjoelund.se on 1 Mar 2016 09:35 UTC Replying to [comment:48 hansolsson]:

However, the optimization of assuming that functions are pure doesn't seem like valid for Modelica 3.2 since impure functions may be called in when-clauses, and there is no way of declaring them pure or impure.

Modelica 3.2 says:

Modelica functions are mathematical functions, i.e. calls with the same input argument values always give the same results. A Modelica function is side-effect free with respect to the internal Modelica simulation state. Specifically, the ordering of function calls and the number of calls to a function shall not influence the simulation state."

So for Modelica 3.2, these optimizations are perfectly reasonable to make.