Large negative pricevalue SE3 2023-05-25 12:00 sent to InfluxDB

[joe-ave]

Hello Olli,

I got a strange large negative value sent to InfluxDB for the hour 2023-05-25 12:00 SE3, it shows -2147483.75 cent/kw. It should have been -0.61 cent/kw. I have not yet checked the Arska-Node, I´ll do that later this evening. At entso-e's site it looks normal.

//Jonas

[Olli69]

Hi Jonas, Thanks for the comment! Did you used the latest code from devel-ui branch? I made some code cleanup with price data (from a array to more general timeSeries object). Main functionality looked good but I did not yet test changes in influx storage. This has not necessarily caused this, specially if the problem is only with one hour. Anyway the old code is still there and can be activated by changing row #define PRICE_SERIES_OLD_DEPRECATED -> #define PRICE_SERIES_OLD.

Problem seem to be with related with ENTSO-E price query. Value -2147483648 (in this case divided with 1000 or so, but not exactly?) indicates missing value. I cannot repeat the problem with SE3 query, but if it has something to do with http 1.1 chunked data, it can happen randomly because chunk size varies.

So I had a quick look, but could to really repeat it yet. Anyway now these erroneous values (indicates missing values) should no be written to influx. I added a check to function update_prices_to_influx .

[joe-ave]

Hello Olli,

It's an older 0.92.0-rc2, it was visible there in the dashboard also. Never seen this behavior before.

But in an other esp32 with a newer 0.99.0-beta1.2662 ( on my desk for testing ) it looks normal so I guess it's somehow related to the older version, I´ll need to upgrade my older board I guess.

//Jonas

[joe-ave]

Hello Olli,

I swapped between different price areas but no luck on SE3, still the same. Trying restart from Admin page did not help. Then I pushed the reset button on the esp32 and now everything is back to normal.

//Jonas

[Olli69]

Hi Jonas, To me it now seems that in the price query response, that hour was split to two data chunks and Arska could not get ,"undestand" price - so it was marked with missing (VARIABLE_LONG_UNKNOWN -2147483648) value.

Resetting made requery, but I do wonder why pressing the restart button didn't help.

Anyway, now in the latest version there is extra check if (current_price != VARIABLE_LONG_UNKNOWN) // do not write undefined values preventing unspecified values written to influx. But I still have to check if there is something to do to make price processing itself more reliable.

Olli

[joe-ave]

Hello Olli,

great that you found a reason to why it happend!

Maybe you could let the Arska-node send another query for that specific hour that have unknown data, and then resend that hour to influxDB? And first maybe just average the price between the hour before and after the unknown hour, that would be least bad and the channel rules would have something good enough to decide against.

//Jonas