Hyundai-Kia-Connect / kia_uvo

A Home Assistant HACS integration that supports Kia Connect(Uvo) and Hyundai Bluelink. The integration supports the EU, Canada and the USA.
MIT License
408 stars 84 forks source link

Kia EV6 - 12 volt battery drain #719

Open manofpants opened 11 months ago

manofpants commented 11 months ago

Not sure how to file this one...

My 6 month old EV6 was found in the morning with a 12 volt (small one, not the big pack for driving) completely dead. It had 92% when I went to sleep and it was unplugged overnight. I was able to get a jump to back it out of my garage and drive it to the dealership to look at by their EV specialist.

The specialist asked me if I had any apps or integrations that connected to the car and the only thing I had was this HA integration. The car has since held a charge in the 12v since disabling the integration.

Was the intermittent ping from HA causing the battery to drain a little bit every time it's off so that in 6 months, the wear from that killed the 12v?

fuatakgun commented 11 months ago

Sorry for this has happened to you.

Having force refresh usually wakes up the car and it draind the battery to some extent. But driving the car day to day should car it back.

But cloud refresh does not impact battery.

We put some warnings around these in readme so users do not put force refresh every minute or so.

manofpants commented 11 months ago

Hey I appreciate that. I did read the warnings and took them to heart so I didn't have it go all that much. I am really only posting this as a curiosity since there's no way to get the logs from the car to determine exactly what caused it. EVs in general seem to have some issues with the 12v battery which I'm sure will get figured out in time, but not yet.

fuatakgun commented 11 months ago

You should have long term statistics of your 12v battery and you should see how it was impacted throughout the night with force update calls.

Do you have a setup of frequent force updates?

IMG_20230908_180029.jpg

manofpants commented 11 months ago

Is that the big battery or the 12v? I reactivated the integration to get this: image

manofpants commented 11 months ago

Oh wow I just realized that's the 12v since the other sensor is "EV Battery Level" which looks like normal charging. OK this is something to dig into...

Looks like it drained down to 80%ish during the day on the 30th image

manofpants commented 11 months ago

Here's the settings image

fuatakgun commented 11 months ago

45 min force refresh, would end up 32 times of calling your car.

Mine is around 2 hours and it is giving enough information. Keep in mind that, car also updates its cloud data regularly without your request. We are also checking if it has been more than 45 min (your config) before asking for new data.

manofpants commented 11 months ago

And also no forcing between like 10 or 11 pm and 5 am so not sure there, either.

cdnninja commented 11 months ago

Lots of reports of these failures from those not using third party as well. Kia seems like want to point this as a third party issue. That being said, unless you have a reason to have your force refresh time that low drop that to say 6 hours and see if you are happy with that.

DJBenson commented 11 months ago

Kia are pointing the finger squarely at users (especially third party apps) for the issue but the issue is a design/electrical issue, people who aren't using third party apps are complaining of battery issues in the UK.

There are two problems I've deduced from various forums;

Waking the system draws a huge amount of current from the battery. I'm not sure if they can fix this part but they would have known when designing the car it would use that amount of power.

More importantly their software is letting the battery drain. The car is capable of (and does) periodically charge the 12v battery, it's just not doing a great job of it. If the traction battery is healthy (above 20% SoC) the car maintains the battery but it also can allow it to run flat even when the traction battery has plenty of charge.

nxenos83 commented 11 months ago

@manofpants I eneded up in a very similay situation -- except I have the default force refresh timing. Depsite longer interval on the force update, I was still seeing change in battery level hourly (dropping about 1% per hour).

I figure either there is a hiccup in the integraiton that resulted in force updates being requested more often OR the car was reporting to the cloud hourly on its own. I had an android auto dongle connected, and I suspect that that was keeping the car awake and reporting to the cloud.

Any chance you also had something plugged into the usb port?

alexw-im commented 11 months ago

There are so many factors that "pointing fingers" is very hard and bound to be easily disproven in any specific case.

When I first started using this plugin, I had the default force interval (4h) and that was too frequent. I switched to basically "never force refresh" (I think I set the max interval between force refreshes of like 13 hours or something, which was the max it'd accept in minutes) and that solved (those particular) issues at the time. Oddly, looking now, I'm back to the default settings of 240min and no refreshes between 22:00 and 06:00.

You have to know a LOT more about the car's power states to make sense of anything though.

Under normal "sleep" circumstances, the cell modem is basically alive drawing a little bit of power looking for SMS commands (probably, I don't know for sure, but AFAIK that's how tesla does it since it's power-cheap) that say things like "wake up and send me data" or "turn on the climate."

When the car gets the "force refresh" command, it wakes up all the OTHER vehicle controllers besides just the telematics module to collect data. All of those modules stay alive drawing ~200 watts for multiple minutes. That consumes a lot of power, and if you do it a lot, it can certainly consume a lot of energy, but it's not the MAIN killer.

The issue beyond that is that the vehicle stays awake in a low but not ultra-low power state for like hours after the command. So by issuing a forced refresh, you're really poking the whole car into a continous higher-draw state for quite a while.

Then finally, on the very long term, there's a long-term storage low-power state where basically only the access control module is on. Even telematics shuts off and the car stops responding to commands. If you go on a long trip and leave the car sitting for months, for instance, it should enter this power state after a few days and draw almost nothing from the 12V battery, and also no longer respond to the app. By issuing forced refresh commands, you keep resetting the timer on entering this low-power state, so long-term, the car might not be able to last through your whole vacation.

In theory none of this should be a problem because the car DOES wake up to recharge the 12V battery from the HV battery when the 12V SOC gets below 80%. In practice, this has issues: 1) it's power-intensive to recharge, because the car needs to wake up and supply ~200W worth of modules for 20min or an hour when all it wants to do is charge the 12v. 2) the logic is designed very naively. The car doesn't want to over-charge a faulty 12V battery that might be dead anyway, thereby over-discharging the main battery. It solves this problem rather stupidly by saying "If I have to wake up a 10th time to recharge the 12V battery, something's wrong, i'll assume it's faulty, and I will stop doing that until it's replaced." All well and good if there IS an internal fault in the battery, but quickly leads to problems if the battery is draining because the car is just awake a lot and there are a bunch of legitimate draws on it.

Finally: I ran into a very interesting edge case the other day myself. I have a dashcam installed in parking mode, but I added a switch to turn OFF parking mode when I don't need it. In one position, camera "power" is tied to an always-on fuse tap, and the camera's "parking mode" signal is tied to a fuse tap that IIRC supplies the infotainment circuit, which is only on when the car is in "accessory" or higher power state. In the second position, both wires are attached to the second fuse tap, such that the camera turns off completely when the car is off.

Well, switching that switch does something weird; I'm not quite sure what. Maybe it shorts the two inputs momentarily? Maybe it's just changing the load on one circuit and the car senses it? Maybe it's back-powering the "off" circuit THROUGH the camera?

Whatever it is, if I switch that switch after the car is off, the infotainment screen starts to power on briefly then goes back to sleep. I recently realized that when it does this, it puts the car silently into a high-drain ~110W power state indefinitely, constantly sucking down the 12V battery. It'll recharge itself over and over, ignoring the 10x "Aux battery saver +" fault timeout I mentioned above, but of course it'll suck down the HV battery at a couple percent a day. I discovered this while away on vacation with the car plugged in - telemetry and my charger were both reporiting massive unexpected power consumption with the car recharging itself from shore power every hour or two.

So long story short, the 12V system in this car is QUITE complicated, perhaps OVER complicated, and also implemented very naively in places. Anything outside the most vanilla "go to work, turn off, come home, plug in" type of usage scenario is liable to upset the balance in ways that often SHOULDN'T be a problem yet somehow are. For instance, this integration, aftermarket 12V accessories, heavy app usage, and so on.

alexw-im commented 11 months ago

Oh and I forgot to mention, one other HUGE "gotcha" when trying to make sense of 12V telemetry in the EV6:

The VCMS logic doesn't try to maintain a full charge on the 12V battery, it tries to maintain like 90% or something.

So notice above when you plugged in, the car battery shot up to 100% immediately, but then started discharging while you were still plugged in and even while the car was still charging. Then after it stopped charging but while STILL plugged in, the 12V battery started discharging down into the mid 90s. Then once you started driving, it discharged FASTER down into the mid 80s.

That's all (paradoxically and I think stupidly) normal.

What should then happen after that point is that if the 12V is below ~90% while driving, it'll charge up until it's around there, then mostly stay level. And if it's above that, it'll actually discharge while driving until it's about level at ~90.

MichT368 commented 11 months ago

IMHO, if the battery was completely dead, this is a faulty starter battery. I had the same thing about 6 weeks after I got my new EV6. They checked everything and then changed the battery. No more problems since 40.000 km. I use the kia_uvo intergation since the beginning and you can rest assured: This is not draining anything. Not even if I have plugged in the wallbox all night long drains the 12V battery. So, force your dealer to exchange the battery, because a fully drained 12V starter battery is not able to fully regenerate after recharging anyways.

alexw-im commented 11 months ago

IMHO, if the battery was completely dead, this is a faulty starter battery. I had the same thing about 6 weeks after I got my new EV6. They checked everything and then changed the battery. No more problems since 40.000 km. I use the kia_uvo intergation since the beginning and you can rest assured: This is not draining anything. Not even if I have plugged in the wallbox all night long drains the 12V battery. So, force your dealer to exchange the battery, because a fully drained 12V starter battery is not able to fully regenerate after recharging anyways.

The system is honestly too complex with too many interactions to make a statement like that based on evidence like that. In a traditional gas vehicle, you'd be right, but in this kind of EV system, cause goes both ways: a dead battery can manifest the other symptoms, and the other symptoms can result in a completely dead battery.

It solves this problem rather stupidly by saying "If I have to wake up a 10th time to recharge the 12V battery, something's wrong, i'll assume it's faulty, and I will stop doing that until it's replaced." All well and good if there IS an internal fault in the battery, but quickly leads to problems if the battery is draining because the car is just awake a lot and there are a bunch of legitimate draws on it.

Because of this system, you can experience a "latching fault" where one particular instance of over-use (maybe you went on vacation and used the app a lot? Maybe you had misconfigured this integration for high-refresh but then set it back?) ends up turning off aux battery saver +, which can then result in a deep discharge and then death of an otherwise previously healthy 12V battery. Then, when the dealership replaces it, in the process ABS+ gets reset and your car goes back to operating properly, recharging the 12V when it gets low and thus never draining it to death.

MichT368 commented 11 months ago

Are you driving the EV6? Have you any possibility in measuring this car every day?

Sorry, but that is my personal practical experience with my EV6 plus the Kia Uvo Integration on HA plus my wallbox since 40.000 km in 1 1/2 year driving experience. And I shared this experience IMHO, which means HUMBLE. And the basic idea was to help @manofpants, and not start a discussion what someone thinks I can say here or not.

I am a car mechanic and have a long time experience with both, PV-systems including Lion-Batteries and the behavior of lead batteries in all kind of cars, plus it's my second EV. I have about 100.000 km EV-km's done. I am monitoring this (and the former) car since the first day in my garage, using not only Home Assistant, but also reading battery data from OBD2. And based on all that knowledge AND personal experience with a faulty produced starter battery in my EV6, I said exactly that what I have written.

And if you would have read a bit more carefully, you could have estimated, that I am exactly knowing, what I am talking about, because not everybody knows about sulfation of a lead battery after going dead. It doesn't help to simply reload the battery and being happy, that the thing has some voltage back. It will drain even faster the next time, which also underlines: "First exchange the starter battery, and then move on."

And yes, if you have a flat tire, you have to repair the flat tire first before you are talking about tire imbalance. Right? Figuratively speaking, you are talking about a tire with an unbalance. I, on the other hand, am trying to help someone who has a flat tire. Do you understand the difference?

Will not answer a second time, because this is OT here anyway. IMHO, this is no software bug. Not in the car and not with the kia uvo integration, which is, by the way, a superb piece of software. Thank you for that!

fuatakgun commented 11 months ago

Thanks all for the discussion and feedback.

We all know that Kia/Hyundai don't allow us to use the app if the car is stationary for a long period of time and they claim that, it is done to preserve the battery.

Moreover, I have some previous discussions on this repository, one user had measured the current drawn from the battery while asking for a force update and it was higher than we all anticipated.

So, to my knowledge, force update consumes a bit of the battery and it needs to be used cautiously. Moreover, it needs to be recharged by the car while driving.

Please keep an eye on your car and yourself. Thanks again

alexw-im commented 11 months ago

I'm sorry if that came off at all as a personal attack; it was certainly not meant to be.

MichT368 commented 11 months ago

...and I reacted too impulsively. I am also sorry about that.

Your post has encouraged me to buy a data logger for my battery so that I can understand more precisely the behavior of the charging unit for the 12V battery when the car is idle. OBD2 is no longer active when the vehicle is switched to inactive mode, so my picture is not complete, even though I can see that my starter battery shows no change in state of charge, even after days without movement in the garage and with interrogation via the Kia Uvo integration every 2 hours and a charging plug inserted.

As you can see, emotions are necessary to get things going. In this sense, you have now brought me a little further and I am grateful to you for that.

Alba56 commented 10 months ago

I don't understand.. a starter battery is designed to give huge current for the starter motor (hundreds of amps) for a very short time. Not designed to give (relatively) small currents (tens of amps) for long periods of time. So why are they using starter batteries in EV:s? EV:s don't have starter motors. Moreover, they show repeated 12v discharge/recharge cycles because of these weird software driven HV->12v recharge 'policies'.

At minimum, there should be a deep cycle AGM battery factory-installed, by default. Or the 'house'/utility type of battery used in boats (they have separate batteries for starting, and for equipment). It would alleviate this problem (a quote from a previous post): "not everybody knows about sulfation of a lead battery after going dead. It doesn't help to simply reload the battery and being happy, that the thing has some voltage back. It will drain even faster the next time, which also underlines: 'First exchange the starter battery, and then move on.'" .

MichT368 commented 10 months ago

Technically, I agree to 100%. Starter-batteries in EV cars are a misconstruction. However, they are cheap and available. And their recycling process is approved and reliable. Whatever we think, when it comes to mass-production, has other dimensions, than simple end-consumers, like we are, have in mind. First of all, you have to find a supplying resource that has the capacity to deliver what you ask for. Second, if you do this in masses, you have to be aware of the costs, And third: You have a responsibility to make things ready for recycling. All this factors in mind, the responsible engineers have to find a compromise between which what would be technically perfect and what's on the market. And that's what we get for now.

Unfortunately, for what I know about charging features of the different battery systems available, you can't simply replace the inbuilt acid batteries with lion ones. Because they have a completely different charging curve. Correct me, if I am wrong.

alexwhittemore commented 10 months ago

Technically, I agree to 100%. Starter-batteries in EV cars are a misconstruction. However, they are cheap and available. And their recycling process is approved and reliable.

I’m not even sure they’re a bad choice, for all the reasons you state and I completely agree with. The first key advantage (high power density) isn’t relevant, but the second (excellent wide temperature performance) still is, and every other kind of design momentum is in favor.

I don’t even think EV’s represent a significant evolution from ICE in terms of low voltage system design. They don’t demand high starting current, but otherwise everything else stays the same. I think the key issue is that EVs are universally expensive and therefore tech-heavy, including lots of constant drains like touch-free access, telematics, monitoring features, and so on.

Back in the 80s, there was zero constant draw on a starter battery. Then there were key fobs and keyless entry systems which added some. Then there were telematics, and best I can tell, all automakers have struggled to balance the user experience of telematics systems with not-destroying starter batteries. My previous car was a 3 series and forum members would whine constantly that telematics went awry and killed their battery, despite that system employing I think smarter mitigations and power design than Kia does.

I just think modern super-tech-heavy vehicles have in some cases done a bad job at the system level designing around the 2^26281736 ways the system might enter a dark corner of unanticipated power states.

Unfortunately, for what I know about charging features of the different battery systems available, you can't simply replace the inbuilt acid batteries with lion ones. Because they have a completely different charging curve. Correct me, if I am wrong.

you’re wrong only in the sense that you can usually spend money to fix compatibility issues like that. Commercial lithium starter batteries build in a whole BMS to take care of the internal cells as necessary even in the presence of a system designed to take care of a lead acid battery. Or a manufacturer could do the opposite and design a charge management system assuming lithium.

but either way you’d be quite right that they’d miss out on the ubiquity and economies of scale of the lead acid market, and pay more for it. Generally not worth it I think.

Of course you can smooth over a lot of the problems by just throwing big guns at them (in the form of an aftermarket lithium starter with like 2x the energy in the same form factor). So in the end, they could have spent money designing better, or could have spent money cheating with a bigger battery, and I think most of us wish they had done one of the two :)

Alba56 commented 10 months ago

Yes, Li-ion needs a software update, and even lead acid AGM needs a higher charging voltage than a normal flooded lead acid, otherwise it reaches only 80% state of charge, I'm told.

Well, lots of good perspectives and viewpoints, appreciate that. Let me add mine (sorry about a lengthy post but I have to..). I've no experience on battery or automotive industry but a >30 years experience in building commercial IT systems, thus I am biased to that (perhaps narrow) view. IT development is, in the end, all about (1) user experience and (2) redundancy.

When we conduct a usability test for a system, we classify the severity of the shortcomings detected in the user interface. The worst kind is classified as 'catastrophic': when the user intended to do some task (possibly a mission critical task), but the shortcoming was so bad that she didn't manage to accomplish it at all. Typically, the guy(s) who created the system say that the failure was 'user's fault', or caused 'unnecessary whining' etc. But if 6-8 of all 10 usability testers experience the same, then the interface has a design flaw.

I have a 17 month old Kia EV6 and I or my wife got stranded in the middle of nowhere, just because of a flat 12v battery (at the same time, the high voltage battery showed 80% full !). That user experience is catastrophic. Far worse than some minor nuisance (refresh rate etc) in some minor comfort connect app in my cell phone. The car is full of electric kWh:s (some 60kWhs, it is worth of over 50 lead acid batteries, 80ah each), and you are stranded - because of missing electricity. Ridiculous. That happened to us, suddenly. The car was switched off, and I sat inside the car for 15 minutes with my key in the pocket and then it didn't start. No Kia connect or whatever. And when it happened again month later, I swapped the Korean 'Delkor Clarios' lead acid to an Exide EFB lead acid.

If I only had carried a jump starter, i.e. had reduncancy ... A small Li-Ion Telwin Drive Mini (14cm x 7.5cm x 1.5cm) would suffice in this case, since no need to 'crank' any engine, just to show 12V to the system, to poke the HV battery to charge the 12V again. I went and bought that immediately.

During my lifetime boating around isolated islands this stranding has never happened. If my starter battery goes flat then I can use my utility/house battery to crank the diesel and get charging the starter again. And vice versa, if I run my house battery flat, then I can still crank the diesel with the starter battery, to start charging my house battery again. 'Jump start' in both directions. Redundancy. Well, if both batteries go flat, I still have redundancy, because it is a sailboat - it moves, and solar panels and a small windmill will help eventually. Lots of confidence. The moon landers had 3 (three) computers doing the same thing.

My wife has lost some confidence with our Kia EV6 just because of this very minor but annoying thing. And Kia is losing some of its credibility because there are lots of stories ('whining') in the forums about 12V battery. That is a very big price for Kia company saving some hundred bucks in a 50k car. The users are considcering to switch to Polestar or whatever. Also, many Tesla owners say that they need to swap their lead acid 12V batteries approximately once in every two years. That is, AFTER their getting-stranded UX, many of them, every so often. For that reason Tesla S and X (not 3 and Y) started using Li-Ion 12v batteries (and adjusted their software accordingly). A very tiny one suffices, even a 0.1kW version, if the charge policy is smart enough, they say. But I wonder if the software checks it in 15 minutes interval, which interval was enough in our EV6 to kill the inferior Korean cheap battery. And 1 or 2 deep discharges will finish it very rapidly (not so in Exide EFB or even Exide AGM).

I have only once bought, and never will buy again, a cheap starter battery to use as a house battery in my boat. Never even comes into my mind any more, no way. It doesn't serve the purpose, navigation and cabin lights go dim much sooner, I'm afraid to use the fridge at all, autopilot goes dead, radio is silent, radar won't work, etc. Ok, I once did that, and it was horrible crap for that purpose. Similarly, in my opinion, you should not use conventional cheap starting batteries in EV:s, just because 'it is a car' not a boat, or because they are cheap, or because your car repair shop doesn't stock anything else, since they have never heard of anything else. That is, UNLESS EV manufacturers get sorted out the current problems maintaining the 12V battery. Lead acid AGM is less than twice the price than a lead acid but provides 3 or 4 times more deep cycles. But I wonder again the 15 minute interval challenge.

Car manufacturers are missing proper redundancy-thinking. If a small jump starter in your pocket will save you from a catastrophic failure (and save the car company losing its credibility), then, why they don't equip the car with an integrated 'jump starter'? A bit similar thinking than in my sailboat. And Tesla: you can't even open a Tesla bonnet if 12v is dead, to access the 12v to wake it up. Opening the hood needs some weird gymnastics: fishing some wires out of the towing hole and connecting an external tiny button-cell battery, if you happen to carry it with you (some hide the button battery into that hole) etc. In EV6 you can mechanically open the door and then the bonnet. I keep my small cheap Li-Ion jump starter in the frunk. That is an absolute must for all EV owners. But it could be designed as integrated.

See also https://insideevs.com/news/546087/tesla-liion-12v-auxiliary-battery/

"The lead-acid batteries turned out to be problematic in EVs, due to a different use case than a starter battery and manufacturers are expected to switch to lithium-ion versions sooner or later (partially, with the potential transition to 48 V auxiliary systems). Some individual users were already replacing their 12 V batteries with a li-ion version on their own"

But no bad feelings, no intents to insult.

Alba56 commented 10 months ago

Let me clarify a bit more, why I personally don't think a traditional cheap lead acid start battery is not a good idea, even if car manufacturers could tackle some of the current problems maintaining the 12V battery by 'smarter' software. Those batteries spend topped up most of their lives, having always good voltage. A short crank doesn't take away much SOC, and is again quite soon topped. Now everybody of us knows, that such a battery doesn't tolerate deep decharges to total empty, so avoid it, and you are good for many years. But NO. If they experience repeated cycle 100% - 50% - 100% over and over again, then they also will die quite soon, in 2 years (as with Tesla experience), because they are not designed for that either. If I used a starter battery in my boat for 'house', its lifetime was two years, while a proper battery (be it lead acid AGM, lead acid Gel etc) for that purpose is lasting 7-10 years.

Now, those 'smarter software' writers of EV car software would perhaps appreciate if the 12v battery would tolerate something else than having to be topped up all the time. You wish at least some relief in the design parameters, i.e. not having to constantly look at the 12v battery, while the car is switched off, offline. Because that constant looking worsens further the situation and it is a vicious circle. The EV car is more like a cell phone. It needs a 12v battery, a working horse, which you can cycle between 100% and say 30%, over and over and over again, with a peace of mind. Something is going on and draining in modern cars, even if the car is switched off, and to calculate the 'peeking' schedule whether everything is still ok (based for instance immediate voltage history), or whether offline charging is again needed (because of Kia Connect, or because of sniffing the key location, or trunk light or lifting trunk door or whatever has consumed the battery in the meantime), is easier in a 100-30-100-30-100 scenario than in 100-95-100-95-100-95 scenario. In the latter, if your software can't possibly match that schedule, then the only alternative left is that the battery get sacrified much sooner than you are used to in conventional cars.

The end result is the 'problematic 12v battery' we are seeing.

scott400k commented 9 months ago

Hi everyone, so I have a (UK based) 38KWH Ioniq and I find the 12V Battery "sensor" only gives reading when the car is in driving mode...is that "normal"?

Thanks

ZuinigeRijder commented 9 months ago

Car manufacturers are missing proper redundancy-thinking. If a small jump starter in your pocket will save you from a catastrophic failure (and save the car company losing its credibility), then, why they don't equip the car with an integrated 'jump starter'? A bit similar thinking than in my sailboat. And Tesla: you can't even open a Tesla bonnet if 12v is dead, to access the 12v to wake it up. Opening the hood needs some weird gymnastics: fishing some wires out of the towing hole and connecting an external tiny button-cell battery, if you happen to carry it with you (some hide the button battery into that hole) etc. In EV6 you can mechanically open the door and then the bonnet. I keep my small cheap Li-Ion jump starter in the frunk. That is an absolute must for all EV owners. But it could be designed as integrated.

A better approach would be to have a fail-safe solution, that at least the 12 volt battery is not damaged due to under-voltage, something like this 12V Battery Low Voltage Cut Off Switch On Protection Undervoltage Controller Under-Voltage Control

This is an undervoltage switch controller that turns off the load power when the voltage drops below the set value. When the battery voltage rises to the set value, the module will automatically restart the load.

freaksdotcom commented 8 months ago

Discharge is related to the value of the scan interval, but that's not the only factor.

On Dec 1, I took the car to the dealer and they charged the 12v and told me that it was fine at 81%.

I re-added the Kia integration to HA to monitor the battery with default scan and refresh intervals.

The 12v held charge close to 80% value over the weekend, but on Dec 5 it started to drop until I drove the car on Dec 8. It dropped at a similar rate.

On Dec 10, I changed the Scan Interval to 180 min (from 30). The slope of discharge flattened slightly.

The last spike in the graph is from this morning - I put the car in Utility Mode and plugged in the L2 charger. I read elsewhere that this will charge the 12v, similar to when driving. However, when the HV battery stopped charging the 12v battery level actually dropped!

image

cdnninja commented 8 months ago

Only force scan should ping the car. Regular scans don't. Default for force scan now is over 24 hours I believe. Other things can keep it from sleeping too.

alexwhittemore commented 8 months ago

I find it extremely interesting (and surprising) that scan interval should have any impact whatsoever on battery level, given that it only pokes the cloud API and shouldn't have any direct impact on the car itself at all.

I notice a couple of things about that graph though: 1) your battery is discharging below 80%. Theoretically, Aux Battery Saver+ should turn on at 80% to top it up. In practice, I notice that sometimes happens as low as like 75%, but nevertheless, your graph indicates that ABS+ has shut itself off. If ABS+ has to activate 10 times in a row (that is, 10 times without the car being driven/turned on/etc in between), it assumes the 12V battery has an internal fault and shuts itself off forever, meaning from that point forward your battery is much more likely to go flat with a normal amount of undriven time. It resets itself after the car's been driven (charging the 12V) for some arbitrary period of time in which the algorithm decideds "never mind, the battery looks healthy to me again, must have been replaced." 2) your graph SHOULDN'T update as frequently as the scan interval. If it does, that implies that your car has been awake the entire time, continuously, sending updated telemetry to the cloud. The frequency of new data in your graph implies that your car is never going to sleep, which would certainly be a root cause for ABS+ to shut itself off.

For example, here's my telemetry. It's a little tough to tell, but you can see from the sharp changes in EV battery which imply I'm driving that the data updates with high frequency while the car is driving, but only updates every couple hours (at the most frequent) when the car is idle. Your graph is updating nearly as frequent as the scan interval.

image

3) you've run into some counterintuitive aspects of charging on that last test there. Utility mode doesn't apply when charging. It only does two things - disable the auto-shutoff timer (which is normally 90min) and immobilize the car (so you can't drive it without power cycling, which requires the key). Both are already true when charging (no auto-off timer and the car is immobilized), so the moment you plugged in, you actually disabled utility mode. Presumably you were already pretty close to your charge limit at the time, so plugging in functionally just turned off the car a few minutes later without you realizing.

On top of that, 12V charging actually behaves a bit oddly while plugged in. This car doesn't float charge the 12V battery for whatever reason. It generally tries to maintain it at like 85-90%. When you plug in to charge the traction battery, it WILL charge the 12V all the way up to 100%, but at some arbitrary point it'll start consuming it again back down to ~85, even if you're still plugged in and charging the traction pack. It'll also immediately stop charging the 12V the moment traction charging ends. You can see in my graph above that at midnight my 12V started oscillating around 99-100% after being fully charged, then about 12:30am this morning, the charge completed and the 12V immediately started draining, despite being plugged in.

In other words, the surest way to charge your 12V to full is actually to leave the car in utility mode UNPLUGGED, but even that's not guaranteed to hold the 12V at 100%.

freaksdotcom commented 8 months ago

your car has been awake the entire time, continuously, sending updated telemetry to the cloud

The only thing that I could think of would be a CarlinKit wireless CarPlay dongle. Do you think that keep the car awake? I unplugged it this morning when I put the can in utility mode.

I attached a Battery Tender to the car overnight. When I checked the tender this morning, its green "Charged" light was on, indicating that it was applying a float charge.

According to the car, the battery was slowly draining overnight and stopped reporting values when it reached 33%.

I've had the car unplugged and in Utility Mode all day. It hadn't started reporting again and requesting an update returned:

Remote commands are not available due to low battery level. Battery must have at least 40% charge to execute remote commands.

I just drove it around the block which seems to have forced it report. 10 hours in Utility Mode charged used 4% of the HV battery and raised the 12v from 33% to 44%

image

image

alexwhittemore commented 8 months ago

The only thing that I could think of would be a CarlinKit wireless CarPlay dongle. Do you think that keep the car awake? I unplugged it this morning when I put the can in utility mode.

I don't think so, and I don't believe I've had the experience myself of that being a problem, but it's certainly possible and I endorse eliminating the variable while you troubleshoot.

I attached a Battery Tender to the car overnight. When I checked the tender this morning, its green "Charged" light was on, indicating that it was applying a float charge.

According to the car, the battery was slowly draining overnight and stopped reporting values when it reached 33%.

I think you might be running into another measurement setup gotcha here. Did you have the tender directly on the battery terminals, as opposed to having the positive on the terminal but the negative on the bodywork of the car?

The 12V harness has an inline current shunt on the negative side between the body and battery, for the sake of measuring all the current into or out of the negative terminal. If you bypassed that with the float charger, the battery management system would "see" that the battery was continuously discharging all night but without the voltage changing.

I can't say your observations make perfect sense to me given that factor, but at any rate, an external tender very well could confuse the SOC estimation. For hypothetical example: maybe ABS+ will only kick on at a certain voltage which was never achieved because of the float charger, but the SOC estimator sees that 66% of expected capacity was drained into loads, ergo SOC must be 33% despite the voltage. In my experience, ABS+ always seems to turn on at 12.59V where indicated SOC isn't actually consistent, so that theory has merit at least.

I just drove it around the block which seems to have forced it report. 10 hours in Utility Mode charged used 4% of the HV battery

The power cost of keeping the electronics all turned on is something like 250W based on my measurement, so 250W*10h=2.5kWh and 2.5/74kWh @100% indicated works out to 3.3%. So that tracks - 4% is exactly what it should cost for 10h of utility mode with some 12V battery charging and no additional drains.

and raised the 12v from 33% to 44%

Again, based on the logic above, I don't know that this measurement is very trustworthy. It sounds like you used the battery tender and had utility mode on simultaneously? That's even more odd. That would really through the charge management system for a loop.

At this stage, what I'd personally do is install a BM2 voltage monitor to keep track of the battery volgage over time, and change my Kia Connect password to at least temporarily disable this integration as well as any other possible third party software from having an impact. If you still see a steep crash from 13V down to 12V in the course of a day, you'll learn there's definitely some continuous, real drain. if you see something that looks like the plot below, you'll know that, when software integrations are disabled, the system actually performs nominally.

image

In this plot, every brief blip down is the car waking up all the various modules to gather and report telemetry - basically a "forced update" whether actually forced from the app/API, or just a scheduled wakeup. The 1h high period at 1AM was ABS+ kicking on to top the 12V up, and the shorter blips are me driving the car. to and from work.

The steep slope starting after the evening drive is abnormal - that's actually me deliberately causing a fault that keeps the car woken up continuously as yours seems to be. If your power-off phases look like that, that's indicative of some fault internal to the car.

freaksdotcom commented 8 months ago

Did you have the tender directly on the battery terminals, as opposed to having the positive on the terminal but the negative on the bodywork of the car?

Yes, I attached it directly to the battery using a ring terminal harness.

It sounds like you used the battery tender and had utility mode on simultaneously?

No, I removed the harness since it hadn't helped. I've put the car on a 15A charger and I'll leave it overnight. The charger measured 12.5V before starting. The negative is clamped to a strut bolt so the current shunt isn't bypassed.

I ordered a BM6 yesterday so I'll be able to see what's going on.

freaksdotcom commented 8 months ago

The 12v battery level has dropped overnight to under 40%, the point where Kia Connect refuses to fetch data.

The charger was in float mode when I checked it this afternoon. I stopped it and checked voltage, which read 12.9V.

The BM6 is currently showing 12.98V and the car is reporting 42% SoC.

Does the car need to be reset or recalibrated for the SoC to work?

alexwhittemore commented 8 months ago

Does the car need to be reset or recalibrated for the SoC to work?

I think you’ve answered that question experimentally - it sure looks like it to me.

I’d disconnect the 12v battery for a few minutes then reconnect it, leave the car alone a few hours, and see what the battery monitor trace looks like.

freaksdotcom commented 8 months ago

Back with some more data.

The car was plugged in at the start of the graph. The BM6 showed 100% SoC and 12.8-13V. Over the next 20 hours the car woke up every 90 minutes. HA was configured to fetch from cache every 180 minutes.

I drove my son to school on 14-Dec, about 20 minutes round trip. Before leaving, HA reported 30% SoC. The BM6 read 47% SoC and 12.53V. Afterwards, HA read 37% SoC, BM6 56% and 12.8V.

When I got home, I changed HA to fetch from cache every 30 minutes. The car started waking up every 60 minutes.

ABS+ is not running. This may be due to the car's SoC calibration being off.

The car is currently in utility mode to try to add some charge before I have to leave the house. I'm going to disconnect the 12V and run a reconditioning cycle on it later today. Hopefully this will recalibrate the car to the SoC and get ABS+ working again.

Voltage (V) SoC(%) image

manofpants commented 8 months ago

Wow I really didn't expect this to garner so much traction, but I love the collaboration trying to figure this stupid thing out.

One new bit of info I discovered today: I drove my kids to school, came home, parked in the garage, turned it off -- car not plugged in and just sitting at 70%. 20 minutes later, I go back into the garage to get something and I hear a fan whirring somewhere. Turns out my AC was running in the car. The car was off. The screens were off. The air went on. I go in and turn the acc/ready and then back to off and the AC stops. An hour later and I go to drive somewhere and guess what? 12v battery dead and needed to jump again.

So...what could've made the AC turn on after the car was off?

freaksdotcom commented 8 months ago

So...what could've made the AC turn on after the car was off?

Do you have any departure times scheduled? They can cause the HVAC to run.

There is a class action investigation related to this issue: https://chimicles.com/hyundai-ioniq-kia-ev6-battery-drain-class-action-investigation/

manofpants commented 8 months ago

Nope. No scheduled anything, never went into the Uvo app today either and I've disconnected the HA integration a while back just in case.

freaksdotcom commented 8 months ago

The HV charger has a cooling fan that may run during charging. Maybe it runs when the HV contactors are closed as well, such as when ABS+ is on?

As @alexwhittemore suggested, get a BM2 or BM6 battery monitor to get some insights. The BM6 is currently $14 on Amazon.

Start gathering data to bring to the dealership, otherwise they're just going to charge your 12V and send you home.

freaksdotcom commented 8 months ago

I just spoke with the manager of my dealer's service department regarding this issue.

Kia requires that the battery fail a health test with their battery tester before it can be replaced.

The battery SoC is stored in the computer and cannot be reset without using Kia's tooling. Disconnecting the battery does not work, and OBDII doesn't either.

His advice is:

  1. Reset your Kia Connect password. He's seen at least two cases where someone's account was compromised and the car was being messed with.
  2. Gather as much data as possible, such as with a battery monitor. (I'd be hesitant to bring up using HA or a 3rd party app since they may point their finger.)
  3. Create a case with Kia Customer Care and attach all supporting evidence. This will give your dealer leverage to replace the battery even if it passed the health test.

The OEM part is a sealed lead acid and Kia will only replace it with another SLA.

alexwhittemore commented 8 months ago

Back with some more data.

The car was plugged in at the start of the graph. The BM6 showed 100% SoC and 12.8-13V. Over the next 20 hours the car woke up every 90 minutes. HA was configured to fetch from cache every 180 minutes.

I drove my son to school on 14-Dec, about 20 minutes round trip. Before leaving, HA reported 30% SoC. The BM6 read 47% SoC and 12.53V. Afterwards, HA read 37% SoC, BM6 56% and 12.8V.

When I got home, I changed HA to fetch from cache every 30 minutes. The car started waking up every 60 minutes.

ABS+ is not running. This may be due to the car's SoC calibration being off.

The car is currently in utility mode to try to add some charge before I have to leave the house. I'm going to disconnect the 12V and run a reconditioning cycle on it later today. Hopefully this will recalibrate the car to the SoC and get ABS+ working again.

Voltage (V) SoC(%) image

Your car is waking up, presumably for a force refresh, nearly once an hour. That’s WAY too frequent. The only way I know of to make that happen is with this integration and the force refresh interval set to 60min.

Perhaps something is stuck about the integration’s config?

freaksdotcom commented 8 months ago

Force Refresh is set to 2880 minutes. The scan interval was 180 minutes until I changed it to 30 minutes as I mentioned above.

I disabled the integration yesterday around 3:30 pm. The car began charging the 12V shortly after 6:30 pm. I re-enabled the integration around 9:30, set the refresh interval to 180 minutes, and plugged the car in. It has not stopped charging the 12V, keeping the voltage ~ 14.8V for the past 16 hours.

In spite of the long charge time, Kia Connect has shown between 78% and 80% SoC.

freaksdotcom commented 8 months ago

Perhaps something is stuck about the integration’s config

Possibly, since disabling and re-enabling it seems to have caused a change in behavior with the wakeup frequency.

It's also possible that by filing a claim with the Kia Customer Care portal, they've pushed some sort of OTA update that is responsible.

More data:

12v charging behavior has been wacky since re-enabling the HA integration Something is very wrong.

The car went into a very long 14.78 V charge cycle starting around 7:00 PM on Dec 15. This lasted until the car was driven at ~ 4:30 PM on Dec 16. After that, it dropped to 14.68 V but continued to charge. There is no way it took over 20 hours to fully charge the battery.

After a while, it has entered a cycle where it charges for 20 minutes, then stops.

During this entire time, the car's telematics never see the battery go over 80% SoC. This entire time, the BM6 is showing the battery at 100% SoC.

This dropped the HV battery to lose ~ 1% charge overnight.

image

SoC(%) vs  Time

Voltage(V) vs  Time

cdnninja commented 8 months ago

Debug logs will also tell you when HA is polling the car and what type of poll it is.

alexwhittemore commented 8 months ago

It's also possible that by filing a claim with the Kia Customer Care portal, they've pushed some sort of OTA update that is responsible.

The architecture of the car doesn't support that - only the infotainment module can be OTA'd, and you'd know about it if it happened.

How many times did you drive the car on the 17th? I agree your HA plot up until 4:30pm Dec16 looks weird and concerning, but from that point on, it looks like your telemetry update rate actually becomes more reasonable. The 12V sticking at ~80% is what the car's logic generally tries to do, so that's not odd in itself. That trace actually looks good to me as long as those EV SOC battery drops are caused by actual driving, and it's not losing 5-10% over the course of the day just sitting there.

The thing that's weird is how all-over-the-place the voltage is throughout the 17th. I'd expect it to look like that if you drove a zillion short trips that day.

freaksdotcom commented 8 months ago

I made two trips on the 17th. One to the store around 2:00pm and another returning home around 3:20pm. (The dip in HV battery around 6:00 is simply because I didn't force an update when I got home.)

After that, the car charged the 12V at around 4:20, 5:20, 6:20, and 7:20. It chilled out a bit until 9:30, then 10:50, 11:20, and 11:50.

The HV battery only lost 1% between 6:00pm on the 17th and today in spite of the charging wierdness.

Debug logs for the last day or so:

2023-12-17 00:29:18.206 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 2.038 seconds (success: True)
2023-12-17 03:29:20.354 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 2.186 seconds (success: True)
2023-12-17 06:29:22.088 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 1.920 seconds (success: True)
2023-12-17 09:24:50.206 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 1.103 seconds (success: True)
2023-12-17 12:24:52.369 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 2.201 seconds (success: True)
2023-12-17 14:09:52.134 DEBUG (MainThread) [custom_components.kia_uvo.services] Call:{'device_id': 'XXXX'}
2023-12-17 14:09:55.132 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 0.763 seconds (success: True)
2023-12-17 14:19:38.025 DEBUG (MainThread) [custom_components.kia_uvo.services] Call:{'device_id': 'XXXX'}
2023-12-17 14:19:39.289 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 0.788 seconds (success: True)
2023-12-17 14:19:39.815 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 0.991 seconds (success: True)
2023-12-17 15:09:12.177 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 0.931 seconds (success: True)
2023-12-17 18:09:14.747 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 2.580 seconds (success: True)
2023-12-17 21:02:23.575 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 0.837 seconds (success: True)
2023-12-18 00:02:25.296 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 2.128 seconds (success: True)
2023-12-18 03:02:27.101 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 1.934 seconds (success: True)
2023-12-18 06:02:02.385 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 0.967 seconds (success: True)
2023-12-18 09:02:04.425 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 2.258 seconds (success: True)
2023-12-18 10:27:16.376 DEBUG (MainThread) [custom_components.kia_uvo.services] Call:{'device_id': 'XXXX'}
2023-12-18 10:27:20.973 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 0.644 seconds (success: True)
2023-12-18 10:29:05.413 DEBUG (MainThread) [custom_components.kia_uvo.services] Call:{'device_id': 'XXXX'}
2023-12-18 10:29:08.474 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 0.804 seconds (success: True)
cdnninja commented 8 months ago

Debug for the api need to be on for real data but that's more for you than me unless you see unexpected behaviour. It will contain lots of sensitive data.

freaksdotcom commented 8 months ago

Debug logging is enabled. Unless there's another location it's logging to, that's all that it's produced.

cdnninja commented 8 months ago

To confirm as per readme the line for Hyundai Kia connect api was added?

freaksdotcom commented 8 months ago

D'oh, I had custom_components.hyundai_kia_connect_api: debug in my config.