Weather Data Sub hourly interpolation

[EttoreZ]

The IBPSA library default option for weather data sub hourly interpolation is the spline approach. While conducting the analysis for the weather bestest #1314, we noticed that when in the weather file two consecutive hourly values were identical, the spline interpolation would lead to an error with respect to the exact interpolation. Therefore this simplified yearly analysis was carried out to compare the Spline, the Fritisch-Butland and the Steffen inteporlation models. The results are shown in the charts below for the weatherdata carried by the weatherbus in terms of % of hourly steps for which there are almost identical successive values, and the yearlt % error with respect to the mean of those two identical values, in the table instead are reported the absolute yearly errors. I also attach a .zip file with the sample .csv output files and the Python script I used to generate the charts.

But taking a look at the results it could be a good option to change the default interpolation method from Spline to the Fritich-Butland.

WeatherInterpolation.zip

[mwetter]

@EttoreZ : Thanks. I will compare the CPU time with the new setting across the models of the Buildings library. The changes to the weather data reader in the IBPSA library are on branch issue1359_weatherDataInterpolation.

[EttoreZ]

I will merge issue1359_weatherDataInterpolation into issue1314_BESTEST_weather to get the updated .json result files after this change is made permanent.

[mwetter]

The computing time is comparable, some models are faster, some slower, but no clear trend. See attached zip file.

branches_compare_dymola.zip

[mwetter]

The model Fluid.Sources.Examples.Outside_Cp shows that the new interpolation gives better results. The plot below shows dotted the old implementation, red the new implementation and green piece-wise linear interpolation.

[Mathadon]

@mwetter @EttoreZ I'd argue that the dotted line is a better interpolation.. The new interpolation causes much larger time derivatives around the table interpolation points, which are in my opinion less physical than the old ones. Seeing these results, I would stick to the old implementation. I also expect the time integrator to have less problems with the more smooth profiles.

[Mathadon]

Some more elaboration. Suppose that this figure showed HDirNor. The definition of HDirNor in the weather file is the average solar irradiation of the 30 minutes before and after the sample point. So the integral of the solar irradiation after interpolation should equal the value in the table (multiplied with 1h). I drew a 1 hour box on top of the figure of Michael and marked the two areas below and above the sampled value. These areas would ideally cancel each other out. That's clearly not the case, but the effect is also clearly larger with the new interpolation method. Since it simply has no overshoot.

[EttoreZ]

@Mathadon I will try to go point by point for what I think after this brief analysis:

• it would seem from the chart that the new interpolation methods have much larger derivatives, but actually they are very close / if not the same with respect to the spline derivatives.
• One could argue though that the flat part of the curve is not strictly physical and the spline smothing copes with that. However, if someone wants to strictly represent the interpolated data as close as possible than the new approach is better.
• following up with the discussion on the image if you think about it the previous timestep data point was 0, while in the interpolation the value is increasing in the second half of the hour, that area corresponds pretty closely to the one highlighted. This means that the area of the spline is actually overshooting, over a long period of time I guess the overshooting/undershootings will cancel each other out. However if someone wants to model a very specific transient that new approach should be better. These are just my two cents after carrying out the analysis and reading the papers from which the methods were derived. let's see what @mwetter thinks.

p.s. in the table above I was comparing the hourly and subhourly results of each methodology with itself, instead I should have compared the results with the real values reported on the weather file. This the new table:

[mwetter]

@EttoreZ @Mathadon : The rectangle of @Mathadon raises a good point: Are we making a systematic error in the morning that cancels in the evening? This would be bad and speak for keeping the old implementation. @EttoreZ : Can you split the error in radiation in the errors from 0 to noon and from noon to midnight?

I have not seen an adverse effect on the solver (see zip file above in this thread). Although the time derivatives change faster with the new method, see figure below at t=6h, I did not observe issues with the solver.

I suggest we check if the error is biased in the morning vs afternoon and then make a decision on how to proceed.

[EttoreZ]

As discussed this morning I made a sample case assuming the radiation as a sin-wave, with the single wave the differences were pretty mild, so I put two sin waves. Taking a look at the chart below, as Filip was mentioning the spline interpolation does get closer to the physical value, so it is not an overshooting. I guess though for other variables which instead are the instaneous values at the timestamp and not an average value over the hour, it could make sense to use an interpolation method that fits more closely the data. This would not be a troublesome implementation either, since the DataReader already divides the radiation data from all the other data due to the time shift.

[Mathadon]

• One could argue though that the flat part of the curve is not strictly physical and the spline smothing copes with that. However, if someone wants to strictly represent the interpolated data as close as possible than the new approach is better.

But this is not the point of the weather data reader. It's goal is to represent the underlying physical phenomenon as closely as possible, which is not a piecewise-linear curve, so you should not try to model it as such.

Are we making a systematic error in the morning that cancels in the evening?

We should check that the integral of our interpolation method output is close to the energy content that is represented by the table values. If at least that matches, a shift a bit to the left or the right won't make a slot of a difference since building envelope models are nearly linear.

[mwetter]

The "shifted interpolation spline" (cyan) matches the true data best. This is what is on the master branch already. I suggest we stick with this for the radiation.

For the other data, I suggest we also stick with what we have now. @EttoreZ do you agree or does your BESTEST validation make a case for a change to Fritsch-Butland, noting that we should stay close to physics and not close to what other tools may (systematically) do similar to each other.

[beutlich]

FYI: Univariate table interpolation of upcoming MSL v4.0.0 features the modified Akima-spline interpolation. See https://github.com/modelica/ModelicaStandardLibrary/pull/3188 for background information.

[EttoreZ]

@mwetter after what @beutlich said, I think it's best to keep the normal spline for now and adopt the modified Akima-spline when it will be available. The modified Akima comes with the adavantages of the spline interpolation (check first chart with a sinousodal wave) giving a more physical behaviour, and the advantages of the piece-wise cubic interpolation avoiding weird oscillations (check the second chart with a step change, this can happen when dealing with sky cover for example).

[mwetter]

@beutlich @EttoreZ : Thanks for the feedback. Let's stick with what is now on the master and revisit when MSL 4.0.0 is used.