[Bug]: SOC deviating from target SOC

MichaKersloot commented 9 months ago

Contact Details

No response

VRM portal ID

c0619ab410ae

Country / region

Netherlands (nl)

B max

8.5

TB max

3

FB max

3

TG max

4.1

FG max

4.1

Battery costs

No response

Buy price

No response

Sell price

No response

feed-in possible

yes

feed-in control

yes

Version

0.1.6

What happened?

Just saw the SOC change for the current hour ( 12:00 ) from 24% to 20% for the remaining 15 minutes in this hour. This made Dess discharge the battery while the hour-price was almost at it's lowest. Now the SOC is set to 26%, so it's now re-charging the energy it just dumped

screencapture-venus-1881-dess-2023-10-03-13_11_01

Relevant log output

No response

Screenshots

No response

khostri commented 9 months ago

Same situation here. I think DESS should check next hour SOC and if next SOC is higher than current hour planned SOC and current SOC is higher than current planned SOC, DESS should not discharge battery but leave it at idle.

dirkjanfaber commented 9 months ago

But probably then we get complaints from people activating dynamic ESS when the prices are high and the system does not discharge. And we probably also get complaints from impatient users that the system does not work (in the first hour) Agreed that the first impressions of a system are important and there is room for improvement, but also note that the system is designed for running for a long time.

khostri commented 9 months ago

I understand. I think described situation is for cases when SOC changes during that hour or system reached slightly higher SOC during charge - like 50% instead of 48% or so. In that case I think more people will notice discharging 2% of SOC to start charging it again in next 10minutes than lost opportunity from selling those 2 percent because price is known and that SOC change is not because of price change but because of some kind of recalculation based on same input values :)

khostri commented 9 months ago

Also if next hour SOC is higher anyway (as i have described) battery will still charge so discharge/charge will just add loses with same result. And battery charging means that price for selling is not high enough and that's why charging is planned

dirkjanfaber commented 9 months ago

Getting back at the system at hand and looking at the gathered data. This does seem to be an actual bug (and not just a case of wrong initialisation).

At the first arrow point, the target SOC was no longer followed. The user switched of Dynamic ESS at that moment, so that can be explained.

The second arrow point shows oscillation. This should be fixed if you switch from firmware 3.10 to the candidate release branch.

The third arrow shows a non-expected deviation of the target SOC. That one might be caused by the same bug from arrow point 2, but could also be caused by another bug. At about 13.50 the third arrow problem disappears and the system follows the target SOC again.

Now comes the hard part, which is reproducing the issue and figuring out what triggered it. @MichaKersloot : Do you recall doing anything to your system around that time or other anomalies?

MichaKersloot commented 9 months ago

That's a good point. At the moment i'm not sure loses are accounted for. Sometimes it looks the battery is charging and discharging to stay at a SOC %. I can imagine this is the case for a few watts, but sometimes it looks like there are going over 500 watts in and out the battery. Although I have to say that part seems to be less the case than it was like 2 weeks ago.

MichaKersloot commented 9 months ago

Ah... @dirkjanfaber our messages just crossed each other ;-) I see the oscillation is recognised and should be fixed.... To be honest, I've upgraded from 0.1.3 (I think...) to 0.1.6 on september 22th. I've 'played' with the ESS settings just before that to try to get my batteries to a real 100% charge (not really succeeded). I've upgraded and haven't touched the victron after that time.

MichaKersloot commented 9 months ago

As I do monitor the victron with Home Assistant via Modbus. Is there any setting / value you would like to see? I also did see the target SOC change from 24 to 20 in the table that's chown at the bottom of the dess dashboard. It looks like this change is not visible in the graphs you have.

dirkjanfaber commented 9 months ago

While we are looking further into it (but we do need to find a way to reproduce the issue in order to find a solution).

You can either choose to update to the latest Venus candidate release (which should fix that second arrow problem and possibly this one too) or stay on 3.10. And let us know if it happens again. I don't think there is a lot more that can be done at the moment.

As I do monitor the victron with Home Assistant via Modbus. Is there any setting / value you would like to see?

We do have all of the vrm logging, so that should be sufficient.

I also did see the target SOC change from 24 to 20 in the table that's chown at the bottom of the dess dashboard. It looks like this change is not visible in the graphs you have.

According to our logs, the target SOC has not been 24 today. But let's not get too distracted right now. My colleague will take a look if he can explain the deviation from the SOC in the coming days. If he can't explain it, we either need to consider this an incident or dig deeper into it.

MichaKersloot commented 9 months ago

Well, that 24 could be 26, but your right... details..

I've just seen another anomaly. The target SOC was 18%, current SOC was 22% and the battery was charging...

SimonYoungtree commented 9 months ago

I also had a similar event today: at 13:40 the SOC was at 45%. Then it started discharging to 41% at 13:58 (no need at all to discharge because price were low). Then at 14:45 it started charging again. This could perhaps be related to next days prices coming on-line by that time?

vrm portal ID: c0619ab31cc4. Software version 3.10 (With separately applied fix by Victron) and NodeRed 1.06

khostri commented 9 months ago

Today I checked this behavior again - turned off DynESS for some time to reach SOC around 89%. DynESS planned SOC was like 79%. After turning on DynESS again, I wanted to check how it works - SOC should reach 100% today to be able to sell in the evening during one hour peak price. But system started to discharge battery now even during very low price, SOC was recalculated after turnign it on again not to reach 100% anymore. And selling price is nearly on the price of battery cost. I have tried to increase battery cost over the selling price, but system still tries to discharge battery to reach planed SOC (-10 % from current) - so it is selling from battery when selling price is lower than battery cost now. Its against making profit target now. For me this issue and issue #78 commented by me (feeding in time of negative price despite setting) is optically biggest mental issue as I see it :)

Now battery cost is set to 0.07, selling price is 0.65 and will be as low as 0.55, but still selling relatively high amount:

EDIT: Next observation - charging/discharging mode is somewhat "hard" I have tried to inject target 98% to SOC 0

system changed, but it went to the opposite extreme - starting to charge not only from solar, but like 3kW from grid. Injecting same value (98%) right in the new hour but into /Schedule/1/soc instead of Schedule/0/soc looks like a lot better - it switched to charging from solar, pushing excess into grid as expected. All those injects are not reflected on Dashboard

dirkjanfaber commented 9 months ago

@khostri: Few points:

this issue is about the SOC deviating from the target SOC. The problem you are describing is not about that. Please open a new issue if you need further help.

But as I am writing this response, I might as well shortly address your concerns:

inaccuracies are being punished on the grid. Running simulations over longer time and lots of systems has proven to be the most cost-effective solution to not being able to reach the target SOC. So this is expected behavior. We are looking at ways to improve that, but for now it is what it is.
if you want the table to show the same value, you should also inject it into the context field (where the table reads it from).

If you want more info, please open a new issue with a clear subject. I will respond to all of the issues eventually, but as there are a lot of people asking for my attention, it will sometimes take a few days before I get to it.

MichaKersloot commented 9 months ago

Hi, seems to be a tricky thing this steering ;-) I've seend (and heard...): I mean the sudden change to 81% at say midnight.

Question. As a part of my house is running on AC-Out, does running the Release Candidate have greater risks on this? I guess not, because that should be handled by the Multiplus itself and not by the GX device?

dirkjanfaber commented 9 months ago

That peak is something we are aware of. It only lasts a short while and not affecting the (dis)charging that much. But it is already on the todo list. And it does not seem to happen only at midnight (though on most systems that midnight peak is bigger than the other ones)

The release candidate vs the regular firmware on the GX device should indeed not affect the AC-Out behavior on the Multiplus.

MichaKersloot commented 9 months ago

In that case, what would be more helpful? Staying on this version the gather extra data in a 'stable' environment? Or update and check if some problems are solved?

dirkjanfaber commented 9 months ago

I guess that running the latest candidate release is best. If it occurs there, we are certain that the bug is still in the code. We've planned on further investigating the cause next week.

MichaKersloot commented 8 months ago

For your information. I've just installed the RC 3.11~2

dirkjanfaber commented 8 months ago

@MichaKersloot: We've been looking into the deviation. It might be caused by the fact that the battery has slightly more capacity than advertised. Do you have some more information on the used battery? Can you try setting the capacity to 9 or 9.5 kWh? That can't harm the system. It might charge and discharge a little quicker, but it should also solve the deviation.

MichaKersloot commented 8 months ago

Hi, i've just changed the setting to the raw capacity of 9.6 Kwh.

dirkjanfaber commented 8 months ago

@MichaKersloot : Can you also update the Node-RED flow to the latest version (0.1.7)

MichaKersloot commented 8 months ago

@dirkjanfaber: done. Do I need to alter the flow, or re-import the example one?

dirkjanfaber commented 8 months ago

No need to do so. The change from the previous version to this one did not change the flow (only the .js file changed, fixing a bug around midnight).

It should remove the annoying peak you see below:

dirkjanfaber commented 8 months ago

Those midnight peaks are gone now, so that is good.

And otherwise, it looks to follow the target SOC well and there are no large deviations (>1%). So if you have no objection, I'll close this issue.

MichaKersloot commented 8 months ago

Thanx for your effort in solving this issue! It does look some strange thing happened at 15:00 on the 16th. There the dess SOC is set to 95%, but the battery stopped charging at 89%. At 16:00 the dess dropped to this 89%. This morning at the 17th, although not in your graph, another issue. The dess SOC is set to 15% at 07:00 and even to 10% at 08:00 but the battery never discharged below 17% during that timeframe. I'm not aware of any other limits than the 10% that should be reserved for a power fail situation.

dirkjanfaber commented 8 months ago

Agree that the 15:00 on the 16th looks unexpected. If you don't mind I'll ignore that one for now.

Regarding not going to the bottom, you have the setting:

Which prevents going below 20%. If I am not mistaken, this is the com.victronenergy.settings, pathSettings/DynamicESS/MinSoc field. That field is about to become deprecated. There are two settings at the moment:

The ESS minsoc com.victronenergy.settings/Settings/CGwacs/BatteryLife/MinimumSocLimit
The DESS minsoc, com.victronenergy.settings/Settings/DynamicEss/MinSoc Currently, Discharge stops at the higher value of the two.

In order to link those already, you can use the following flow:

[{"id":"fbce6d35fe3e1aa8","type":"victron-input-custom","z":"b9bfd7d55fb5c6e4","service":"com.victronenergy.settings","path":"/Settings/CGwacs/BatteryLife/MinimumSocLimit","serviceObj":{"service":"com.victronenergy.settings","name":"com.victronenergy.settings"},"pathObj":{"path":"/Settings/CGwacs/BatteryLife/MinimumSocLimit","name":"/Settings/CGwacs/BatteryLife/MinimumSocLimit","type":"number"},"name":"","onlyChanges":false,"x":410,"y":720,"wires":[["e25f1acc1d3799b5"]]},{"id":"e25f1acc1d3799b5","type":"victron-output-custom","z":"b9bfd7d55fb5c6e4","service":"com.victronenergy.settings","path":"/Settings/DynamicEss/MinSoc","serviceObj":{"service":"com.victronenergy.settings","name":"com.victronenergy.settings"},"pathObj":{"path":"/Settings/DynamicEss/MinSoc","name":"/Settings/DynamicEss/MinSoc","type":"number"},"name":"","onlyChanges":false,"x":1000,"y":720,"wires":[]}]

MichaKersloot commented 8 months ago

Oh, I wasn't aware of that setting. 'Strange' the schedular doesn't take this setting in account and tries to lower the soc beneath this setting.

dirkjanfaber commented 8 months ago

In the algorithm the value already got deprecated, so that only uses the ESS minsoc.

victronenergy / dynamic-ess