search

helgeerbe / OpenDTU-OnBattery

Software for ESP32 to talk to Hoymiles Inverters and Victrons MPPT battery chargers (Ve.Direct)
GNU General Public License v2.0
298 stars 63 forks source link

[Request] Add adjustable interval time for powermeter http request #486

Closed ntfrnd closed 2 months ago

ntfrnd commented 12 months ago

Is your feature request related to a problem? Please describe.

I see in the console that the HTTP request is made every 10 sec. to read the (Tasmota-) powermeter. This seems very slow to me, especially since the request via web browser is easily possible every second.

Describe the solution you'd like

It would be helpful to make the request interval adjustable for the http(s)+json - powermeter from e.g. 1....30 sec. In one second steps.

Describe alternatives you've considered

No response

Additional context

No response

spcqike commented 12 months ago

it should be possible to make a forced update of your powermeter, within the powerlimiter loop.

if you want, you can try to compile your own firmware.

steps to do: https://github.com/helgeerbe/OpenDTU-OnBattery/blob/116da051146df9f5714cb18a2895703d426fb3d0/src/PowerLimiter.cpp#L431

change this line to int32_t newPowerLimit = round(PowerMeter.getPowerTotal(true));

this will force the getPowerTotal function to make a forced Update https://github.com/helgeerbe/OpenDTU-OnBattery/blob/116da051146df9f5714cb18a2895703d426fb3d0/src/PowerMeter.cpp#L97-L107

so, instead of only giving the last stored powermeter value, it will query your powermeter and update the values beforehand. this only works for SDM1/SDM3 and HTTP, but shouldn't make problems with MQTT or SML.

philippsandhaus commented 12 months ago

There is actually a possibility to configure the interval, it seems however not to be exposed via the web interface. The easiest way for you would be to make a backup of the configuration (config.json) edit it and restore it. The interesting part is the following ("interval":10):

...
"powermeter": {
    "enabled": true,
    "verbose_logging": true,
    "interval": 10,
    "source": 3,
    "mqtt_topic_powermeter_1": "",
    "mqtt_topic_powermeter_2": "",
    "mqtt_topic_powermeter_3": "",
    "sdmbaudrate": 9600,
    "sdmaddress": 1,
    "http_individual_requests": false,
...

But I think it would make sense to make this be configurable via the web interface.

ntfrnd commented 12 months ago

@philippsandhaus Thank you for the tip 👍. For the first trial it is a good possibility to test it. Furthermore it would be great to integrate it in the web interface.

spcqike commented 12 months ago

@philippsandhaus well, having done this myself some month ago, i already forgot that its possible 😄

if one implements a changeable interval, wouldn't it be a better option to just enable or disable "forced update"? as this would always give the nearest possible reading. as long as the DTU querys the powermeter randomly/ independent from powerlimiter, one still can have an offset.

philippsandhaus commented 12 months ago

Hmm, I think this would harm more than it would help. The parameter interval really makes sense to fine-tune how often the respective source is polled. If it is too often OpenDTU as well as the requested source might get unresponsive. For example I am using a Hitchi Powermeter reader running on ESP8266 with Tasmota and am polling it via HTTP. Even with 10s I often run into timeouts. But on the other side If the interval is too big the requested values might be too old for DPL.

If you are using "forced update" this interval parameter is basically meaningless and you have no chance fine-tune it.

spcqike commented 12 months ago

i just tried it and, as for a short test, i dont see any disadvantages.

[DPL::loop] * ENTER ** 12:12:54.876 > [DPL::loop] battery interface disabled, SoC: 0 %, StartTH: 0 %, StopTH: 0 %, SoC age: 1128 s 12:12:54.970 > [DPL::loop] dcVoltage: 39.20 V, loadCorrectedVoltage: 39.20 V, StartTH: 0.00 V, StopTH: 0.00 V 12:12:55.073 > [DPL::loop] StartTH reached: no, StopTH reached: no, inverter is producing 12:12:55.177 > [DPL::loop] SolarPT enabled, Drain Strategy: 1, canUseDirectSolarPower: no 12:12:55.376 > [DPL::loop] battery discharging allowed, PowerMeter: 231 W, target consumption: -20 W 12:12:55.484 > [DPL::setNewPowerLimit] requested: 315 W, last limit: 315 W, diff: 0 W, hysteresis: 10 W, age: 58152 ms 12:12:55.585 > [DPL::loop] * Leaving PL, calculated limit: 315 W, requested limit: 315 W (kept last requested) 12:12:56.191 > [DPL::loop] * ENTER ** 12:12:56.297 > [DPL::loop] battery interface disabled, SoC: 0 %, StartTH: 0 %, StopTH: 0 %, SoC age: 1129 s 12:12:56.400 > [DPL::loop] dcVoltage: 39.20 V, loadCorrectedVoltage: 39.20 V, StartTH: 0.00 V, StopTH: 0.00 V 12:12:56.608 > [DPL::loop] StartTH reached: no, StopTH reached: no, inverter is producing 12:12:56.707 > [DPL::loop] SolarPT enabled, Drain Strategy: 1, canUseDirectSolarPower: no 12:12:56.815 > [DPL::loop] battery discharging allowed, PowerMeter: 228 W, target consumption: -20 W 12:12:56.914 > [DPL::setNewPowerLimit] requested: 312 W, last limit: 315 W, diff: 3 W, hysteresis: 10 W, age: 59657 ms 12:12:57.017 > [DPL::loop] * Leaving PL, calculated limit: 312 W, requested limit: 315 W (kept last requested) 12:12:58.182 > [DPL::loop] * ENTER ** 12:12:58.347 > [DPL::loop] battery interface disabled, SoC: 0 %, StartTH: 0 %, StopTH: 0 %, SoC age: 1131 s 12:12:58.450 > [DPL::loop] dcVoltage: 39.30 V, loadCorrectedVoltage: 39.30 V, StartTH: 0.00 V, StopTH: 0.00 V 12:12:58.553 > [DPL::loop] StartTH reached: no, StopTH reached: no, inverter is producing 12:12:58.760 > [DPL::loop] SolarPT enabled, Drain Strategy: 1, canUseDirectSolarPower: no 12:12:58.858 > [DPL::loop] battery discharging allowed, PowerMeter: 227 W, target consumption: -20 W 12:12:58.966 > [DPL::setNewPowerLimit] requested: 311 W, (re-)sending limit: 311 W 12:12:59.167 > [DPL::loop] * Leaving PL, calculated limit: 311 W, requested limit: 311 W (updated from calculated) 12:12:59.580 > [ 1131.670] DPL: waiting for a power limit command to complete 12:12:59.778 > PowerMeterClass: TotalPower: 227.10 12:13:01.793 > [ 1135.709] DPL: waiting for the system to settle 12:13:03.971 > PowerMeterClass: TotalPower: 231.80 12:13:04.492 > [ 1138.665] DPL: waiting for sufficiently recent inverter data 12:13:07.557 > [ 1141.274] DPL: waiting for sufficiently recent power meter reading 12:13:08.763 > PowerMeterClass: TotalPower: 228.00 12:13:08.891 > [DPL::loop] * ENTER ** 12:13:09.117 > [DPL::loop] battery interface disabled, SoC: 0 %, StartTH: 0 %, StopTH: 0 %, SoC age: 1142 s 12:13:09.294 > [DPL::loop] dcVoltage: 39.30 V, loadCorrectedVoltage: 39.30 V, StartTH: 0.00 V, StopTH: 0.00 V 12:13:09.407 > [DPL::loop] StartTH reached: no, StopTH reached: no, inverter is producing 12:13:09.497 > [DPL::loop] SolarPT enabled, Drain Strategy: 1, canUseDirectSolarPower: no 12:13:09.599 > [DPL::loop] battery discharging allowed, PowerMeter: 228 W, target consumption: -20 W 12:13:09.715 > [DPL::setNewPowerLimit] requested: 313 W, last limit: 311 W, diff: 2 W, hysteresis: 10 W, age: 11313 ms 12:13:09.809 > [DPL::loop] * Leaving PL, calculated limit: 313 W, requested limit: 311 W (kept last requested) 12:13:09.911 > [ 1142.904] DPL: the system is stable, the last power limit is still valid 12:13:10.018 > [DPL::loop] * ENTER ** 12:13:10.217 > [DPL::loop] battery interface disabled, SoC: 0 %, StartTH: 0 %, StopTH: 0 %, SoC age: 1143 s 12:13:10.316 > [DPL::loop] dcVoltage: 39.30 V, loadCorrectedVoltage: 39.30 V, StartTH: 0.00 V, StopTH: 0.00 V 12:13:10.421 > [DPL::loop] StartTH reached: no, StopTH reached: no, inverter is producing 12:13:10.528 > [DPL::loop] SolarPT enabled, Drain Strategy: 1, canUseDirectSolarPower: no 12:13:10.623 > [DPL::loop] battery discharging allowed, PowerMeter: 228 W, target consumption: -20 W 12:13:10.695 > [DPL::setNewPowerLimit] requested: 313 W, last limit: 311 W, diff: 2 W, hysteresis: 10 W, age: 11637 ms 12:13:10.837 > [DPL::loop] * Leaving PL, calculated limit: 313 W, requested limit: 311 W (kept last requested) 12:13:10.913 > [DPL::loop] * ENTER ** 12:13:11.040 > [DPL::loop] battery interface disabled, SoC: 0 %, StartTH: 0 %, StopTH: 0 %, SoC age: 1143 s 12:13:11.241 > [DPL::loop] dcVoltage: 39.30 V, loadCorrectedVoltage: 39.30 V, StartTH: 0.00 V, StopTH: 0.00 V 12:13:11.344 > [DPL::loop] StartTH reached: no, StopTH reached: no, inverter is producing 12:13:11.442 > [DPL::loop] SolarPT enabled, Drain Strategy: 1, canUseDirectSolarPower: no 12:13:11.569 > [DPL::loop] battery discharging allowed, PowerMeter: 228 W, target consumption: -20 W 12:13:11.752 > [DPL::setNewPowerLimit] requested: 313 W, last limit: 311 W, diff: 2 W, hysteresis: 10 W, age: 12218 ms 12:13:11.857 > [DPL::loop] * Leaving PL, calculated limit: 313 W, requested limit: 311 W (kept last requested) 12:13:11.995 > [DPL::loop] * ENTER ** 12:13:12.071 > [DPL::loop] battery interface disabled, SoC: 0 %, StartTH: 0 %, StopTH: 0 %, SoC age: 1144 s 12:13:12.158 > [DPL::loop] dcVoltage: 39.30 V, loadCorrectedVoltage: 39.30 V, StartTH: 0.00 V, StopTH: 0.00 V 12:13:12.263 > [DPL::loop] StartTH reached: no, StopTH reached: no, inverter is producing 12:13:12.365 > [DPL::loop] SolarPT enabled, Drain Strategy: 1, canUseDirectSolarPower: no 12:13:12.467 > [DPL::loop] battery discharging allowed, PowerMeter: 233 W, target consumption: -20 W 12:13:12.547 > [DPL::setNewPowerLimit] requested: 318 W, last limit: 311 W, diff: 7 W, hysteresis: 10 W, age: 13518 ms 12:13:12.631 > [DPL::loop] * Leaving PL, calculated limit: 318 W, requested limit: 311 W (kept last requested) 12:13:13.495 > [DPL::loop] * ENTER ** 12:13:13.591 > [DPL::loop] battery interface disabled, SoC: 0 %, StartTH: 0 %, StopTH: 0 %, SoC age: 1146 s 12:13:13.659 > [DPL::loop] dcVoltage: 39.30 V, loadCorrectedVoltage: 39.30 V, StartTH: 0.00 V, StopTH: 0.00 V 12:13:13.741 > [DPL::loop] StartTH reached: no, StopTH reached: no, inverter is producing 12:13:13.815 > [DPL::loop] SolarPT enabled, Drain Strategy: 1, canUseDirectSolarPower: no 12:13:13.899 > [DPL::loop] battery discharging allowed, PowerMeter: 233 W, target consumption: -20 W 12:13:13.971 > [DPL::setNewPowerLimit] requested: 318 W, last limit: 311 W, diff: 7 W, hysteresis: 10 W, age: 14776 ms 12:13:14.104 > [DPL::loop] * Leaving PL, calculated limit: 318 W, requested limit: 311 W (kept last requested) 12:13:15.642 > [DPL::loop] * ENTER ** 12:13:15.744 > [DPL::loop] battery interface disabled, SoC: 0 %, StartTH: 0 %, StopTH: 0 %, SoC age: 1147 s 12:13:15.847 > [DPL::loop] dcVoltage: 39.30 V, loadCorrectedVoltage: 39.30 V, StartTH: 0.00 V, StopTH: 0.00 V 12:13:15.948 > [DPL::loop] StartTH reached: no, StopTH reached: no, inverter is producing 12:13:16.027 > [DPL::loop] SolarPT enabled, Drain Strategy: 1, canUseDirectSolarPower: no 12:13:16.149 > [DPL::loop] battery discharging allowed, PowerMeter: 231 W, target consumption: -20 W 12:13:16.255 > [DPL::setNewPowerLimit] requested: 316 W, last limit: 311 W, diff: 5 W, hysteresis: 10 W, age: 16272 ms 12:13:16.353 > [DPL::loop] * Leaving PL, calculated limit: 316 W, requested limit: 311 W (kept last requested) 12:13:16.462 > PowerMeterClass: TotalPower: 230.50 12:13:16.661 > [DPL::loop] * ENTER ** 12:13:16.767 > [DPL::loop] battery interface disabled, SoC: 0 %, StartTH: 0 %, StopTH: 0 %, SoC age: 1148 s 12:13:16.868 > [DPL::loop] dcVoltage: 39.30 V, loadCorrectedVoltage: 39.30 V, StartTH: 0.00 V, StopTH: 0.00 V 12:13:16.968 > [DPL::loop] StartTH reached: no, StopTH reached: no, inverter is producing 12:13:17.086 > [DPL::loop] SolarPT enabled, Drain Strategy: 1, canUseDirectSolarPower: no 12:13:17.274 > [DPL::loop] battery discharging allowed, PowerMeter: 231 W, target consumption: -20 W 12:13:17.382 > [DPL::setNewPowerLimit] requested: 316 W, last limit: 311 W, diff: 5 W, hysteresis: 10 W, age: 17364 ms 12:13:17.464 > [DPL::loop] * Leaving PL, calculated limit: 316 W, requested limit: 311 W (kept last requested) 12:13:17.582 > [DPL::loop] * ENTER ** 12:13:17.650 > [DPL::loop] battery interface disabled, SoC: 0 %, StartTH: 0 %, StopTH: 0 %, SoC age: 1149 s 12:13:17.730 > [DPL::loop] dcVoltage: 39.30 V, loadCorrectedVoltage: 39.30 V, StartTH: 0.00 V, StopTH: 0.00 V 12:13:17.810 > [DPL::loop] StartTH reached: no, StopTH reached: no, inverter is producing 12:13:17.905 > [DPL::loop] SolarPT enabled, Drain Strategy: 1, canUseDirectSolarPower: no 12:13:17.946 > [DPL::loop] battery discharging allowed, PowerMeter: 224 W, target consumption: -20 W 12:13:18.094 > [DPL::setNewPowerLimit] requested: 309 W, last limit: 311 W, diff: 2 W, hysteresis: 10 W, age: 18789 ms 12:13:18.197 > [DPL::loop] * Leaving PL, calculated limit: 309 W, requested limit: 311 W (kept last requested)

as you can see, the normal power meter loop isn't used, as long as powerlimiter requests the ac power regularly. only after the 60s re-send of the limit and the needed settle time ( 9seconds, is this normal?), you can see the powermeterloop output, in my case every 5s, as i changed the interval with the config, as you described.

12:12:59.167 > [DPL::loop] Leaving PL, calculated limit: 311 W, requested limit: 311 W (updated from calculated) 12:12:59.580 > [ 1131.670] DPL: waiting for a power limit command to complete 12:12:59.778 > PowerMeterClass: TotalPower: 227.10 12:13:01.793 > [ 1135.709] DPL: waiting for the system to settle 12:13:03.971 > PowerMeterClass: TotalPower: 231.80 12:13:04.492 > [ 1138.665] DPL: waiting for sufficiently recent inverter data 12:13:07.557 > [ 1141.274] DPL: waiting for sufficiently recent power meter reading 12:13:08.763 > PowerMeterClass: TotalPower: 228.00 12:13:08.891 > [DPL::loop] ENTER **

on the other hand, calculating your needed inverter power, based on a, probably 10s old meter value, can be inaccurate.

Getting the AC power right before the calculation would probably be more accurate. But I think at the end of the day it doesn't make much difference if you take 5-10s old values or "live values". you probably only give away or save a few Wh. and, since you have control times and inaccuracies anyway (the AC measurement is also always only a rough snapshot) it makes all the less difference.

@schlimmchen, is there a reason why the power limiter loop cycles once every 0.3-1.5 seconds? if we, like now, only use the existing, "old" powermeter value, wouldnt it be enough, to only update the powerlimit whenever powermeter was updated?

if i shortn my log to just these lines

12:13:09.715 > [DPL::setNewPowerLimit] requested: 313 W, last limit: 311 W, diff: 2 W, hysteresis: 10 W, age: 11313 ms 12:13:10.695 > [DPL::setNewPowerLimit] requested: 313 W, last limit: 311 W, diff: 2 W, hysteresis: 10 W, age: 11637 ms 12:13:11.752 > [DPL::setNewPowerLimit] requested: 313 W, last limit: 311 W, diff: 2 W, hysteresis: 10 W, age: 12218 ms 12:13:12.547 > [DPL::setNewPowerLimit] requested: 318 W, last limit: 311 W, diff: 7 W, hysteresis: 10 W, age: 13518 ms 12:13:13.971 > [DPL::setNewPowerLimit] requested: 318 W, last limit: 311 W, diff: 7 W, hysteresis: 10 W, age: 14776 ms 12:13:16.255 > [DPL::setNewPowerLimit] requested: 316 W, last limit: 311 W, diff: 5 W, hysteresis: 10 W, age: 16272 ms 12:13:17.382 > [DPL::setNewPowerLimit] requested: 316 W, last limit: 311 W, diff: 5 W, hysteresis: 10 W, age: 17364 ms 12:13:18.094 > [DPL::setNewPowerLimit] requested: 309 W, last limit: 311 W, diff: 2 W, hysteresis: 10 W, age: 18789 ms

you can see, that the whole calculation and transmission is based on the powermeter value (or the difference based on the hysteresis). in my case, it can change every powerlimiter-cycle (as i force the update now). for most users, the powermeter value is the same for powermeter_interval, isnt it?

spcqike commented 11 months ago

update from my site: i use said "forced update", running smoothly for 4day 8hours now. grafik

only thing i increased is powermeter update time to 2500ms, which is still less than my 5s interval which i set in the config file, or the standard 10s interval. https://github.com/helgeerbe/OpenDTU-OnBattery/blob/116da051146df9f5714cb18a2895703d426fb3d0/src/PowerMeter.cpp#L97-L107

i used 1000ms in my first test but got restarts every now and than. i dont know if the restarts where fixed by the 2.5s interval, or by some other changes/updates, that where merged in the meantime between my two tests.

schlimmchen commented 11 months ago

is there a reason why the power limiter loop cycles once every 0.3-1.5 seconds?

Sure there's a reason. But explaining it in detail will cost me an hour of my time :wink: Please have a look at the code. There are two important mechanisms at play: (1) The loop is "skipped" if any of the plenty reasons (see code) is found to be true. Like the DPL is off (very basic) or there is a power limit command pending to the inverter so it makes no sense to calculate a new limit now (more sophisticated). (2) There is a backoff implemented. In case a new power limit was set, the backoff is small, so we re-calculate a new power limit soon in case it does not fit or in scenarios where the parameters (sunlight, household consumption) changes often.

if we, like now, only use the existing, "old" powermeter value, wouldnt it be enough, to only update the powerlimit whenever powermeter was updated?

I disagree. I can think of at least one other very important input to the DPL which must cause a new limit calculation: Changes in the charge controller output power. We really should not wait until the power meter reading changes to adjust the power limit if the charge controller output plummets due to a cloud or something.

Please tell me how you managed to "quote" the code so beautifully.

spcqike commented 11 months ago

I just select the lines needed and copy a permalink from the source code file

428D0FEF-50BB-4094-966E-92CCCCE8B154

https://github.com/helgeerbe/OpenDTU-OnBattery/blob/116da051146df9f5714cb18a2895703d426fb3d0/src/PowerMeter.cpp#L97-L107
Fribur commented 9 months ago

https://github.com/helgeerbe/OpenDTU-OnBattery/blob/116da051146df9f5714cb18a2895703d426fb3d0/src/PowerLimiter.cpp#L431

change this line to int32_t newPowerLimit = round(PowerMeter.getPowerTotal(true));

PowerMeter.getPowerTotal() and PowerMeter.getPowerTotal(true) is exactly the same, as the header file specifies the default (=forceUpdate parameter omitted) as “True”. So I do not understand this suggestion?

spcqike commented 9 months ago

@Fribur as you can see, right 2 lines after the quote ends, the first implementation of this "force" wasn't true by default. this was changed later.

see below

schlimmchen commented 9 months ago

What are you talking about, @spcqike? Why are you citing the implementation when the default value for the parameter is (always) defined in the header (or forward declaration in general)?

Have a look at https://github.com/helgeerbe/OpenDTU-OnBattery/commit/fd58ad2003a9ad9c55dfab962aa09575badced0c where the flag was introduced. It's default value has always been true, and the existing method calls were not updated.

spcqike commented 9 months ago

well, you are right. i must have mixed this up.

but, i proposed a default "false" to have a better backwards compatibility. https://github.com/helgeerbe/OpenDTU-OnBattery/discussions/246#discussioncomment-6182991

maybe that's what i had in mind 😕

schlimmchen commented 3 months ago

Please have a look at #1077, where setting the PowerMeter polling interval for all providers that do poll (including HTTP+JSON) is configurable through the web UI. It is expected to be part of the next release.

github-actions[bot] commented 1 month ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new discussion or issue for related concerns.