2.1.0 does not update prices over MQTT

[tronde-ams]

ESP32 / 2.1.0 / aidon 3p IT / Norway.

One ESP32 with 2x DS18B20, one without sensors. GPIO 16

Prices are not updated as raw MQTT. I have tried both full and minimal. I have not tried MQTT as JSON. WEB GUI and /energyprice.json are correct.

[gskjold]

Do I interpret this correctly that this only occurred at midnight? Or does it happen more often?

[tronde-ams]

The prices shown above did not change after I updated to 2.1.0. It was not just at midnight. I updatet wery short time after the new code was published.

The prices did change when new prices was fetced today, but not after. .

This is from 15:23 when the correct price is 2,41 (multiplier = 1.25 and NO1) as shown in the /energyprice.json but wrong in the MQTT data.

[ujhede]

While at the subject of prices via MQTT. Is it correct that Prices Sensors are not created in Home Assistant mode ?

[tronde-ams]

Prices did not change until midnight. New prices are wrong.

[gskjold]

@ujhede Correct, there are no price sensors defined in the code. Seems that this was not covered in the initial implementation

[gskjold]

@tronde-ams Found the problem, currently testing a fix

EDIT: firmware attached esp32.zip

[tronde-ams]

First impression is that e009b4e is OK.

I installed just before 15:00.

I will report back after midnight.

[tronde-ams]

Mostly OK by now, but some strange result for raw full format.

It can be that emqx and mosquitto handles things differently. I use two different brokers because that was most convenient for my android app but I will change if you tell me what to test for

Emqx is OK

but mosquitto does not delete all old values. It has deleted 23 and 24, but not after that.

[tronde-ams]

Both units have fetched new prices, and both are deleting old prices as they should do. The trouble can have been because both units was updated after 14:00 yesterday, and the mosquitto unit some hours after the emqx unit so some prices from earlier had been stuck.

Your new code appears to be OK.

[tronde-ams]

Still strange things going on...

Everything was OK until midnight. Both units updated prices and deleted old.

The mosquitto unit started to show old prices again (25 and older) while the EMQX unit was still OK.

I understand next to nothing about how these procesesses are meant to work, so I can only tell what I see.

Since the error repeated from yesterday, I choose switch the MQTT format on both units to see if the problem is related to mosquitto.

As of now EMQX receives MQTT raw minimal and mosquitto raw full.

Nothing changed, but maybe a new ftching of prices and a new midnight will tell more. I will report back.

Edit: I see now that there is more wrong in the OK part for mosquitto. see next post.

[tronde-ams]

Two similar units with same conguration for prices, and different prices shown ??? Mosquitto seems to be correct. I must have happened when or after I switched the MQTT.

This unit is EMQX

* This unit is Mosquitto

[gskjold]

I have adjusted the code so that it will clear all unknown prices instead of just the first one.

As for the difference in price between your two units, I do not yet understand how that could happen. I will have to set up a test with two units to compare.

[tronde-ams]

I have updated one unit to 2.1.1. The unit which ended up showing corrupt prices seems to have gone haywire. I can flash it with USB, and the flasher says hash is OK, but then nothing. It shows up as 192.168.4.1 when I scan the AMS2MQTT network, so it is some kind of life.

The remaining unit is set to push raw full to mosquitto, since that was the initial problem.

I have one spare unit but I must prepare some hardware first.

Please tell me if you want me to test something specific.

[tronde-ams]

I wiped the flash on the corrupted unit, flashed it to 2.1.1, and it is now back on air. I have two units for testing, so tell me what to do with them.

I modified the saved config file from the other unit (changed IP and hostname) and uploaded it to the newly flashed unit. It did not use the saved API-key. I think I have seen this before too, but not worried about it. It can be you shuld check if it is a bug.

[tronde-ams]

At 01:50 it seems like it is OK.

Both units (one raw full and one raw minimal) behave the same way.

The only thing I notice, is that before midnight deleted prices does not show up in the list,,

. while they show up as 0 after midnight, with the exception of the deleted value from the most recent hour being omitted.

I don't think this is a problem. I just tell what I see. .

[gskjold]

Good catch, I think I see why that happened, firmware attached.

As for the API key not being loaded from config file, I cannot see a reason from the code, but I will test that later.

esp32.zip

[tronde-ams]

Is this the correct file? It shows as 2.1.0 and the files are from 2022-03-30

[gskjold]

Hah, I tricked myself, here you go.

esp32.zip

[tronde-ams]

Testing now. Will report back after midnight.

[tronde-ams]

a0d3632

Raw full seems to be Ok.

Some strange result for raw minimal, but it can be this is not a problem, but a result of old prices not being cleared on the broker? It was carried over when I updated. I will report back when new prices have been fetched.

[gskjold]

Weird, it should have cleared them.. Maybe I'm doing it wrong, will investigate.

[tronde-ams]

Still nutters for raw minimal. I have set the mosquitto unit to push raw minimal as well now, to see if it behaves the same way.

[tronde-ams]

I have been testing more. MQTT raw minimal sent to test.mosquitto.org seems to be OK, just as raw full.

The module which has been connected to EMQX shows some strange behaviour. It can be defective in a way, but the strangeness is only related to prices. I will try to find a replacement tomorrow.

It would be nice if someone else could try MQTT raw minimal to broker.emqx.io. It can be I am chasing ghosts...

[gskjold]

Interesting stuff, and thank you for thoroughly testing this. I can't see that the topic for prices should behave any different in minimal vs full, it always publish the same data. But there is a chance this could be related to MQTT buffer size maybe, not sure, gussing.

Any difference in length of topic? Special characters? Assuming it uses SSL? I've never tried these cloud based brokers. Just need some info to set up a proper test environment when the time comes.

Hopefully in Easter I will have some time to look more into this.

[tronde-ams]

I think your a0d3632 code is OK for the problem with update of prices. Make it public.

Something strange seems to be going on with prices and two of my modules now. I have two completely different modules. One of them is the unit with reduced clock speed. This module is running 220124.2 and has been stable for more than two months, but last night it started to corrupt prices just as the other module running a0d3632 has done a couple of times. At the moment it seems to be OK. It did fetch new prices, and updates as it should at the moment. I kept the old code because it was stable, and I used it as a refernece.

The a0d3632 went haywire, and I had to wipe the flash before it accepted to be load flash from USB. Before it went haywire, it behaved the same way as the 220124.2 which has different hardware, so I am a little puzzled.

I t seems like the prices had become stuck in memory, even when I deactivated and reactivated the API-key. I would have expected it to read and use new prices form the API and not from memory? This error was also in the WEB GUI, so not directly related to MQTT.

I find it strange that two different modules with two different firmwares seems to suffer from such a strange error. Do the modules self destruct after some time, ot can it be some kind of memory leak corrupting the flash?

This is a different error, but it seems like something is evolving over time. I need more time to figure out how to test this.

Regarding MQTT:

I use only normal letters and numbers in the topic. 9 characters in total.

I don't use SSL. This was initially set up as a test so I could learn more about MQTT, and since I don't have a local broker at the moment I tried some of the public brokers.