Translation of transmission line model is very slow

[paultjevdh]

Description

The translation stage of building of a 10 line transmission line takes very long compared to smaller models. Enclosed are 2 models Test_T_line.zip

The smaller model (Test_T_line4) takes less than 3 seconds to translate (before compilation starts). The larger model (Test_T_line10) takes up to 16 seconds for translation only. There seems to be a serious performance bottleneck in translation of the transmission line.

Version and OS

• OpenModelica Version 1.22.3
• Versions of used Modelica libraries: MSL version 4.0.0 (but the current 4.1.0 beta is not better)
• OS: Windows 11, 64 bit

[perost]

The smaller model has 1309 equations/variables while the larger has 5446, so the larger model is approximately four times larger. So that seems about right, many things in the compiler scales linearly with the size of the model. The new backend we're working on might scale better, depends on the model structure, but it doesn't work with these models yet unfortunately.

[paultjevdh]

When increasing the # of lines to 20, the nr of equations increases to over 18000 (dymola, I can't find this in openmodelica this fast). In dymola the numbers are: 1279, 5416, 18311 for 4, 10 and 20 lines respectively. The translation times in openmodelica are <3sec, 16 sec, 120 sec. This is slower than linear with # of equations.

The main issue is that translation of this model is relatively slow. Dymola translates and compiles this 20-line model in 12 sec. Or is it simply because this model is unusually large?

[casella]

@paultjevdh, this transmission line model includes mutual inductance couplings between each pair of lines, so its number of variables and equations grows as O(N^2). In fact 18311/5416 = 3.38, which is roughly (20/10)^2 = 4. It's a bit less because only some of the equations are O(N^2), some are O(N), but as N grows, the O(N^2) part eventually dominates.

The ratio of number of equations between line10 and line4 is 5416/1278 = 4.2. If you compile with -d=execstat, you can get a breakdown of the time taken by each phase of the model translation, which is very instructive.

On my PC, the line4 model is translated into C code in 3.4 s, while the line10 model is translated in 25.9 s. The ratio is 7.9, which is clearly larger than 4.2. You can then check for the time spent by each step, and see which ones are multiplied by 4 (hence O(N)) and which are multiplied by O(N^2).

A quick comparison between the two execstat logs reveals that most steps have a time ratio of 4, so they scale as O(N), N being the number of equations.

analyzeInitialSystem (which fails, btw) has a ratio of 8. symbolicJacobian scales with a ratio of 16, hence O(N^2), and so do simCode: created initialization part, simCode: created simulation system equations, and Templates.

Bottom line: some parts of the code generation process scale in a superlinear way.

In 2015 I wrote this paper, where I introduced the ScalableTestSuite library. In the following years, we have tested it extensively in OpenModelica, which allowed to find out many steps that scaled badly because of naive implementation.

We fixed the worst ones, see e.g. https://trac.openmodelica.org/OpenModelica/ticket/4146, so if you check the current performance you can see that in most cases the whole code generation phase scales as O(N).

A few issues remained, but at some point we figured out that it didn't really make sense to first flatten large models made by arrays to scalars and them make these steps more efficient on large scalar models - it's much better to avoid scalarizing the model in the first place and doing the steps on arrays, which can have O(1) complexity. So we stopped working on making the old backend more efficent and started working on a new array-preserving frontend, and a new array-preserving backend.

The new frontend is done.

The new backend is still far from maturity, but we are starting to get promising results on some really large models, see, e.g. LargeTestSuite_NB. Compare this to the results with the performance of the old backend LargeTestSuite_OB. I'm not sure Dymola can handle such very large models, maybe you can try and report. Of course these models are a bit dumb, who may want to simulate a spring-mass system with 1 million equations. But it still gives some useful indications.

Bottom line: we expect the new backend to be able to handle most (if not all) Modelica models in 2-3 years, at that point we'll have a dramatic improvement in compile time performance. If you find a big bottleneck in one of the step, for a model that is relevant for one of your applications, we could look into that, profile it and try to see if there is some obvious naivete in the implementation that can be fixed easily. Otherwise, I guess you'll have to be patient 😅

[casella]

BTW, you can count the number of equations of the line20 model in OMC by just checking the model. This should be quite fast. Compiling it is another story.

[paultjevdh]

Thanks for the explanation. I have noticed that openmodelica is much faster already that it was 3 years ago, so I know things are moving in the right direction.

[casella]

I thought again why some parts of code generation scale as O(N^2). It could be due to the fact that the inductor coupling part is dense, so the Jacobians have N^2 elements. If we generate the code to compute them symbolically, this could explain it. Once more, as soon as we have array-based handling, this could become O(1) in terms of code generation.