search

Hyundai-Kia-Connect / kia_uvo

A Home Assistant HACS integration that supports Kia Connect(Uvo) and Hyundai Bluelink. The integration supports the EU, Canada and the USA.
MIT License
440 stars 88 forks source link

Timestamp of "last_updated_at" sensor is wrong #931

Closed johny-mnemonic closed 1 month ago

johny-mnemonic commented 1 month ago

Region and Brand of car Europe Hyundai Kona 2020

Version of the integration 2.26.6

Describe the bug I have weird issue with a timestamp of the last_updated_at senzor. Most probably some timezone issue. As I live in CET it now translates to +2, but here is what I am getting today:

2024-09-12 17:59:03.516 DEBUG (SyncWorker_11) [hyundai_kia_connect_api.KiaUvoApiEU] hyundai_kia_connect_api - Received forced vehicle data: {'retCode': 'S', 'resCode': '0000'
2024-09-12 17:59:06.467 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 0.417 seconds (success: True)

kona_last_updated_at.last_updated: 2024-09-12 15:59:06.468164+00:00
kona_last_updated_at.state: 2024-09-12T13:59:03+00:00
now: 2024-09-12 18:20:00.757938+02:00

Here I called the force update at 17:59, it succeeded, but in the UI it tells me the data were last updated at 15:59. Almost as if the integration somehow shifted the time based on my TZ, thinking it got the time from the server in CET, while it got it in GMT.

Here you can see server clearly defines the TZ as GMT:

hyundai_kia_connect_api - LoginFormSubmit https://eu-account.hyundai.com/auth/realms/euhyundaiidm/login-actions/authenticate?session_code=XXXXXXXXXXXXXX-XXXXXXXXXX&execution=cb33-4625-b9f1-13fd5d3f3cfa&client_id=XXXXXXXXX&tab_id=XXXXXXX - Response 302 - {'Date': 'Thu, 12 Sep 2024 15:58:21 GMT'

Could this be some new change in the API? Did it return the time in local time instead of GMT previously?

Maarten69 commented 1 month ago

I have same issue.

artursulkowski commented 1 month ago

Europe, Hyundai Kona 2020, Version of the integration 2.26.6

From my logs analysis - invalid timestamp appears when it is getting cached vehicle status. Timestamps are kind of random - sometimes they are from the past, but the next one is even more from the past. What I've noticed - during capturing the information during charging - state of charge was kept stable (same value or increased) - the issue is only with timestamp.

Here is a snippet of my log (not a full JSON to keep privacy) Log times are in CET ( UTC+2) - watch to 'time' in the json

2024-09-11 20:20:11.124 DEBUG (SyncWorker_52) [hyundai_kia_connect_api.KiaUvoApiEU] hyundai_kia_connect_api - get_cached_vehicle_status response: {'retCode': 'S', 'resCode': '0000', 'resMsg': {'vehicleStatusInfo': {'vehicleLocation': {'coord': {'lat': 0.1234, 'lon': 0.1234, 'alt': 0, 'type': 0}, 'head': 0, 'speed': {'value': 0, 'unit': 0}, 'accuracy': {'hdop': 0, 'pdop': 0}, 'time': '20240911192007'},...

2024-09-11 20:50:12.170 DEBUG (SyncWorker_50) [hyundai_kia_connect_api.KiaUvoApiEU] hyundai_kia_connect_api - get_cached_vehicle_status response: {'retCode': 'S', 'resCode': '0000', 'resMsg': {'vehicleStatusInfo': {'vehicleLocation': {'coord': {'lat': 0.1234, 'lon': 0.12347, 'alt': 0, 'type': 0}, 'head': 0, 'speed': {'value': 0, 'unit': 0}, 'accuracy': {'hdop': 0, 'pdop': 0}, 'time': '20240911172007'},....

cdnninja commented 1 month ago

Could this have created it? https://github.com/Hyundai-Kia-Connect/hyundai_kia_connect_api/pull/592

Easier for someone in EU to figure this one out.

artursulkowski commented 1 month ago

@cdnninja thanks for the hint. I will try troubleshoot if it had any impact

artursulkowski commented 1 month ago

Could this have created it? https://github.com/Hyundai-Kia-Connect/hyundai_kia_connect_api/pull/592 Checked code before that pull request: hyundai_kia_connect_api=3.22.0 (with kia_uvo 2.26.0) I have the same issue, looks like something changed on BlueLink servers. At this glance at the code - processing of this last_updated_at is very much different depending on location so rather not reproducible out of EU.

steps56 commented 1 month ago

Use "Force Update" instead of just "update". It "wakes up" the car (do not overuse !) Works like a dream (Europe), see 2 Screens. Bevor and after using Force Update:

Screenshot_20240918_025917_Home Assistant Screenshot_20240918_030003_Home Assistant

yaml for button card in dashboard (device id xxxed):

show_name: true show_icon: true type: button tap_action: action: call-service service: kia_uvo.force_update data: device_id: xxxxxxxxdummyxxxxxc target: {} name: Force update icon: mdi:car-connected show_state: true icon_height: 50px hold_action: action: none

So, for me its working like a charm.

cdnninja commented 1 month ago

Last updated is last time car provided data not last time we checked the servers. Force update prompts it to update from the car vs cached from server. Car reports data for a variety of reasons. Force update consumes car battery.

steps56 commented 1 month ago

As I wrote ... 😎

johny-mnemonic commented 1 month ago

@steps56 You probably missed, that in the OP I wrote I have used the "force update" 😉

I am not sure what caused it as I have seen it only once since I reported the issue. Usually it's fine. So I wonder why it is not consistently wrong or consistently right 🤔 It is not a big issue so unless someone has some lead what might be causing it I am not sure how much investment it deserves.

cdnninja commented 1 month ago

Only thing to note is incorrect time will cause the integration to incorrectly calculate force refresh interval. This will cause it to query more often and consume 12v battery. Worth keeping an eye on it.

johny-mnemonic commented 1 month ago

Sure, but it will also manifest by getting rate limited by Hyundai servers as the amount of force updates is limited (20 per day if I am not mistaken).

fuatakgun commented 1 month ago

if forced update fails, we fallback to cached data, right? can you actually confirm looking at different data if forced update worked? you can always look at Data Sensor attributes to get the actual information btw.

steps56 commented 1 month ago

if forced update fails, we fallback to cached data, right? can you actually confirm looking at different data if forced update worked? you can always look at Data Sensor attributes to get the actual information btw.

If you ask me or in general: Yes Data changes in sekonds if I use Force Update. (as shown at the 2 screens some post before) Absolutly top working integration at the moment 👌.

Main problem about bad connection to the car is bad mobil connection of its sim card to the internet/cloud/server... or even car sits some days unused going to deep sleep. This can only be waked up by driving some time.

johny-mnemonic commented 1 month ago

Got it again now after calling force update:

2024-09-20 15:25:04.450 DEBUG (SyncWorker_18) [hyundai_kia_connect_api.KiaUvoApiEU] hyundai_kia_connect_api - Received forced vehicle data:
2024-09-20 15:25:07.455 DEBUG (SyncWorker_18) [hyundai_kia_connect_api.KiaUvoApiEU] hyundai_kia_connect_api - _get_location response: {'retCode': 'S', 'resCode': '0000', 'resMsg': {'gpsDetail': {'coord': {'lat': XX.929711, 'lon': XX.008683, 'alt': 283, 'type': 0}, 'head': 188, 'speed': {'value': 0, 'unit': 0}, 'accuracy': {'hdop': 0, 'pdop': 1}, 'time': '20240920132506'}
2024-09-20 15:25:07.677 DEBUG (SyncWorker_6) [hyundai_kia_connect_api.VehicleManager] hyundai_kia_connect_api - Time differential in seconds: 7203.677076
2024-09-20 15:25:07.778 DEBUG (SyncWorker_6) [hyundai_kia_connect_api.KiaUvoApiEU] hyundai_kia_connect_api - get_cached_vehicle_status response: {'retCode': 'S', 'resCode': '0000', 'resMsg': {'vehicleStatusInfo': {'vehicleLocation': {'coord': {'lat': XX.929711, 'lon': XX.008683, 'alt': 283, 'type': 0}, 'head': 188, 'speed': {'value': 0, 'unit': 0}, 'accuracy': {'hdop': 0, 'pdop': 1}, 'time': '20240920152506'}
2024-09-20 15:25:08.194 DEBUG (MainThread) [custom_components.kia_uvo.coordinator] Finished fetching kia_uvo data in 0.519 seconds (success: True)

Notice the two different times. There is 'time': '20240920132506' in the get_location response but 'time': '20240920152506' in the get_cached_vehicle_status response 😮

Seems like the location data is the source of trouble...again 😅

steps56 commented 1 month ago

I can't see any problems at my place/car/BlueLink/HA... To get all data in sync in same seconds, I just push the "Force Update" Button and about 5 Seconds later the "Update" Button at my "Ioniq5 Dashboard". That's it, all data are imediately in sync to the local time. Screen shows it (actualized about 10 Min ago to Handy time at top 😉):

Screenshot_20240920_221931_Home Assistant

May be setting time at all stuff may solve the time gap ifthere is any at somebodys setups 🤷🏼‍♂️

cdnninja commented 1 month ago

To confirm, your last updated sensor in HA reverts to the location response time stamp? Location time is placed in the location timestamp here:

https://github.com/Hyundai-Kia-Connect/hyundai_kia_connect_api/blob/05c9d4bb725109ef5359242d32e0c60f7cb91b36/hyundai_kia_connect_api/ApiImplType1.py#L300-L320

In turn placed into the object here:

https://github.com/Hyundai-Kia-Connect/hyundai_kia_connect_api/blob/05c9d4bb725109ef5359242d32e0c60f7cb91b36/hyundai_kia_connect_api/Vehicle.py#L382-L386

I don't believe we expose vehicle.location_last_updated_at in HA only last_updated_at.

I am not clear if this is actually an issue or not? Last_updated_at sensor in HA should match the cached call data response from the server, not last time a call was placed.

steps56 commented 1 month ago

To confirm, your last updated sensor in HA reverts to the location response time stamp? Location time is placed in the location timestamp here:

https://github.com/Hyundai-Kia-Connect/hyundai_kia_connect_api/blob/05c9d4bb725109ef5359242d32e0c60f7cb91b36/hyundai_kia_connect_api/ApiImplType1.py#L300-L320

In turn placed into the object here:

https://github.com/Hyundai-Kia-Connect/hyundai_kia_connect_api/blob/05c9d4bb725109ef5359242d32e0c60f7cb91b36/hyundai_kia_connect_api/Vehicle.py#L382-L386

I don't believe we expose vehicle.location_last_updated_at in HA only last_updated_at.

I am not clear if this is actually an issue or not? Last_updated_at sensor in HA should match the cached call data response from the server, not last time a call was placed.

Hmm, don't understand the programming stuff... But its working nice at my setup. So its not any problem by integration I think. If sombody has problems by getting actual time after using "Force Update" and after that, hitting "Update", those problems may be by the cars bad mobil connection or whatever. I just may confirm, its working perfect at my Home Assistant Setup and car/place.

Crosstest if someone sees any problems by using the Bluelink App. If there is latecy or mismatch or even "try later" message, its bad mobile connection and/or car sittig unused a few days... or mobile white spot at that place. All happend sometimes in the past (and a few days ago by Hyundai Server problems discussed here too at github).

johny-mnemonic commented 1 month ago

@steps56 why do you post here if you don't have issues? 🧐 My data are also in sync once the force update succeeds. The only issue is, that sometimes the last_updated_at is 2 hours in the past (as if it was in GMT and not my TZ). Definitely not related to any trouble with car GSM connectivity.

@cdnninja I am not sure I read the code correctly, I am just a poor SysAdmin not Coder, but you are probably right. The following comment shows the author of the code was well aware that these two timestamps can differ:

last_updated_at and location_last_updated_at can be different.
The newest of those 2 can be computed by the caller.

It just caught my eye when going through the debug log. I struggle to find difference in the logs between the case when the last_updated_at is correct and when it is wrong.

cdnninja commented 1 month ago

@johny-mnemonic I am a sysadmin too but primary coder of this. :)

Yes two timestamps can differ. I don't see a spot where the variables get mixed up. After the above two links where the values are stored to vehicle.location_last_updated_at and vehicle.last_updated_at we display this data here:https://github.com/Hyundai-Kia-Connect/kia_uvo/blob/cd73b7424a63c13095fec3ae47345209303c1d21/custom_components/kia_uvo/sensor.py#L69-L74

with key being the variable off the vehicle object.

You can see all spots last_update_at is touched here:

https://github.com/search?q=repo%3AHyundai-Kia-Connect%2Fhyundai_kia_connect_api%20vehicle.last_updated_at&type=code

Keep in mind as previously mentioned a cached update call will only update last_update_at if the car has self reported data back since last cached update call. Common reasons for the car to report back data is being driven. If parked an unused I wouldn't expect it to report any data back. As such last_update_at will start to age. The force update logic configured in your settings trigger how old this data can be before it wakes the car up using a force update call - which also consumes 12v power.

The reference to location last updated confuses me on how it relates to a timezone issue. Also hard for me to understand without knowing the 1. current time in local time. 2. log data time. 3. home assistant displayed time. 4. Mobile app displayed last updated time. And how those all relate. This would need to be done with no force calls executed as that sounds like brings it to current as we expect it to.

Hope this helps!

Steven77nl commented 1 month ago

I'm having the same issues here. This morning I kept my eyes on the value of the field for a couple of hours after moving my car forcing it to report an update.

I moved my car at 8:09am (UTC+2). I did not do any forced updates since, not via the IOS App, not via the integration.

The IOS App is showing last updated correctly as 08:09am (correct time in my time zone) The HA Integration is setting the last_updated_at entity sometimes to the correct time and sometimes with exactly 2 hours difference.

entity_id,state,last_changed sensor.niro_last_updated_at,2024-09-23T04:09:33+00:00,2024-09-23T06:23:27.213Z (local time 8:23 UTC+2) sensor.niro_last_updated_at,2024-09-23T06:09:33+00:00,2024-09-23T07:38:28.994Z (local time 9:38 UTC+2) sensor.niro_last_updated_at,2024-09-23T04:09:33+00:00,2024-09-23T08:08:30.476Z (local time 10:08 UTC+2) sensor.niro_last_updated_at,2024-09-23T06:09:33+00:00,2024-09-23T08:38:32.045Z (local time 10:38 UTC+2) sensor.niro_last_updated_at,2024-09-23T04:09:33+00:00,2024-09-23T09:08:32.462Z (local time 11:08 UTC+2)

@cdnninja

  1. current time in local time (car updated at 08:09 UTC+2 / 06:09 UTC)
  2. log data time (see logs)
  3. home assistant displayed time (set to GMT+1 Amsterdam, with currently summer time enabled)
  4. Mobile app displayed last updated time (shows the correct local time: 08:09)

Hope this helps to find the issue. If need to provide more logging, let me know.

steps56 commented 1 month ago

Again tested a few minutes ago. If you look at the time send by integrations entity: sensor.ioniq_5_marion_steps_last_updated_at (your name will be differnt for sure) you see this: Screenshot_20240923_123041_Home Assistant if you have used before "Force Update" and a few seconds later "Update". It shows correct time... and if still not, use "actualize" Button at Entity: Screenshot_20240923_123114_Home Assistant I don't use this because no need to see always exactly last update time, was just a proof of working right time by the entity/Integration. Anybody may test same way now. If its not giving the right time (same as Smartphone and HA and...), its may be a problem I never see. Wrong set time at phone or whatever ???

Steven77nl commented 1 month ago

Forced update is fine. Cached update is showing wrong result about 50% of the time.

And I do use the 'Last Updated At' on my dashboards, and then it doesn't look nice it's jumping between the correct time and a 2 hours earlier time.

To help the developers look into this I did another two hour test in which i didn't do a forced update, only cached updates every 15 minutes while having the debug logging turned on.

My actions and observation timeline in local times:

in the attached debug logs you can see this as well, the get_cached_vehicle_status response is sometimes getting time': '20240923123000' and sometimes time': '20240923103000'.

'20240923123000' seems to be the UTC+2 time (my local time in the Netherlands) '20240923103000' is the UTC time.

It looks like the server is sometimes unclear in which time zone to format the date/time. Though the IOS app was showing the correct time all the time during these two hours.

debug_logs.log

johny-mnemonic commented 1 month ago

@steps56 what kind of issue do you have with this sensor? I don't understand it from your posts full of pictures 🧐

@Steven77nl thanks for the observation. I also noticed that it is sometimes randomly showing wrong and correct time. I would swear that I have seen correct time in the HA App on the phone while it was showing wrong time in the browser.

steps56 commented 1 month ago

The pictures/screens just shows what I do to get shown correct time. Background actualisation is most time some hours back... that's normal/the usual behavior and has been never any problem (I see no problem at all, just showing how to get time synchronisized... if someone needs it for whatever reason here and / but stating it as a bug... but no bug for me, sorry)

cdnninja commented 1 month ago

@Steven77nl excellent data and summary. Thank you for that work. The key finding is here. It shows two cached calls. This raw data response shows the data going back in time. This is the raw server data, not adjusted by us. This puts it in a bind as the data is incorrect. The time stamp is attached to the location section, not the car data however only one timestamp exists so not much we can do. 

2024-09-23 13:21:55.273 DEBUG (SyncWorker_51) [hyundai_kia_connect_api.KiaUvoApiEU] hyundai_kia_connect_api - get_cached_vehicle_status response: {'retCode': 'S', 'resCode': '0000', 'resMsg': {'vehicleStatusInfo': {'vehicleLocation': {'coord': {'lat': *******, 'lon': *******, 'alt': 0, 'type': 0}, 'head': 329, 'speed': {'value': 0, 'unit': 0}, 'accuracy': {'hdop': 0, 'pdop': 0}, 'time': '20240923123000'}, 'vehicleStatus': {'airCtrlOn': False, 'engine': False, 'doorLock': True, 'doorOpen': {'frontLeft': 0, 'frontRight': 0, 'backLeft': 0, 'backRight': 0}, 'trunkOpen': False, 'airTemp': {'value': '02H', 'unit': 0, 'hvacTempType': 1}, 'defrost': False, 'acc': False, 'evStatus': {'batteryCharge': False, 'batteryStatus': 100, 'batteryPlugin': 0, 'remainTime2': {'etc1': {'value': 65535, 'unit': 1}, 'etc2': {'value': 560, 'unit': 1}, 'etc3': {'value': 145, 'unit': 1}, 'atc': {'value': 0, 'unit': 1}}, 'drvDistance': [{'rangeByFuel': {'evModeRange': {'value': 383, 'unit': 1}, 'totalAvailableRange': {'value': 383, 'unit': 1}}, 'type': 2}], 'reservChargeInfos': {'reservChargeInfo': {'reservChargeInfoDetail': {'reservInfo': {'day': [9], 'time': {'time': '1200', 'timeSection': 0}}, 'reservChargeSet': False, 'reservFatcSet': {'defrost': False, 'airTemp': {'value': '00H', 'unit': 0, 'hvacTempType': 1}, 'airCtrl': 0, 'heating1': 0}}}, 'offpeakPowerInfo': {'offPeakPowerTime1': {'starttime': {'time': '1200', 'timeSection': 0}, 'endtime': {'time': '1200', 'timeSection': 0}}, 'offPeakPowerFlag': 0}, 'reserveChargeInfo2': {'reservChargeInfoDetail': {'reservInfo': {'day': [9], 'time': {'time': '1200', 'timeSection': 0}}, 'reservChargeSet': False, 'reservFatcSet': {'defrost': False, 'airTemp': {'value': '00H', 'unit': 0, 'hvacTempType': 1}, 'airCtrl': 0, 'heating1': 0}}}, 'reservFlag': 0, 'ect': {'start': {'day': 0, 'time': {'time': '0000', 'timeSection': 0}}, 'end': {'day': 0, 'time': {'time': '0000', 'timeSection': 0}}}, 'targetSOClist': [{'targetSOClevel': 80, 'dte': {'rangeByFuel': {'evModeRange': {'value': 299, 'unit': 1}, 'totalAvailableRange': {'value': 299, 'unit': 1}}, 'type': 2}, 'plugType': 0}, {'targetSOClevel': 100, 'dte': {'rangeByFuel': {'evModeRange': {'value': 384, 'unit': 1}, 'totalAvailableRange': {'value': 384, 'unit': 1}}, 'type': 2}, 'plugType': 1}]}, 'batteryPreconditioning': False, 'batterySoh': 0}, 'ign3': False, 'hoodOpen': False, 'transCond': True, 'steerWheelHeat': 0, 'sideBackWindowHeat': 0, 'tirePressureLamp': {'tirePressureLampAll': 0, 'tirePressureLampFL': 0, 'tirePressureLampFR': 0, 'tirePressureLampRL': 0, 'tirePressureLampRR': 0}, 'battery': {'batSoc': 81, 'batState': 0, 'sjbDeliveryMode': 0, 'batSignalReferenceValue': {'batWarning': 65}, 'powerAutoCutMode': 2}, 'lampWireStatus': {'stopLamp': {'leftLamp': False, 'rightLamp': False}, 'headLamp': {'headLampStatus': False, 'leftLowLamp': False, 'rightLowLamp': False, 'leftHighLamp': False, 'rightHighLamp': False, 'leftBifuncLamp': False, 'rightBifuncLamp': False}, 'turnSignalLamp': {'leftFrontLamp': False, 'rightFrontLamp': False, 'leftRearLamp': False, 'rightRearLamp': False}}, 'windowOpen': {'frontLeft': 0, 'frontRight': 0, 'backLeft': 0, 'backRight': 0}, 'smartKeyBatteryWarning': False, 'fuelLevel': 0, 'washerFluidStatus': False, 'breakOilStatus': False, 'sleepModeCheck': True, 'time': '20240923123001', 'remoteWaitingTimeAlert': {'remoteControlAvailable': 1, 'remoteControlWaitingTime': 168, 'elapsedTime': '00:03:08'}, 'systemCutOffAlert': 0, 'tailLampStatus': 0, 'hazardStatus': 0}, 'odometer': {'value': 1880, 'unit': 1}}}, 'msgId': '0a0e7cd0-799e-11ef-84cc-6408cfce1f83'}

2024-09-23 13:36:56.245 DEBUG (SyncWorker_32) [hyundai_kia_connect_api.KiaUvoApiEU] hyundai_kia_connect_api - get_cached_vehicle_status response: {'retCode': 'S', 'resCode': '0000', 'resMsg': {'vehicleStatusInfo': {'vehicleLocation': {'coord': {'lat': *******, 'lon': *******, 'alt': 0, 'type': 0}, 'head': 329, 'speed': {'value': 0, 'unit': 0}, 'accuracy': {'hdop': 0, 'pdop': 0}, 'time': '20240923103000'}, 'vehicleStatus': {'airCtrlOn': False, 'engine': False, 'doorLock': True, 'doorOpen': {'frontLeft': 0, 'frontRight': 0, 'backLeft': 0, 'backRight': 0}, 'trunkOpen': False, 'airTemp': {'value': '02H', 'unit': 0, 'hvacTempType': 1}, 'defrost': False, 'acc': False, 'evStatus': {'batteryCharge': False, 'batteryStatus': 100, 'batteryPlugin': 0, 'remainTime2': {'etc1': {'value': 65535, 'unit': 1}, 'etc2': {'value': 560, 'unit': 1}, 'etc3': {'value': 145, 'unit': 1}, 'atc': {'value': 0, 'unit': 1}}, 'drvDistance': [{'rangeByFuel': {'evModeRange': {'value': 383, 'unit': 1}, 'totalAvailableRange': {'value': 383, 'unit': 1}}, 'type': 2}], 'reservChargeInfos': {'reservChargeInfo': {'reservChargeInfoDetail': {'reservInfo': {'day': [9], 'time': {'time': '1200', 'timeSection': 0}}, 'reservChargeSet': False, 'reservFatcSet': {'defrost': False, 'airTemp': {'value': '00H', 'unit': 0, 'hvacTempType': 1}, 'airCtrl': 0, 'heating1': 0}}}, 'offpeakPowerInfo': {'offPeakPowerTime1': {'starttime': {'time': '1200', 'timeSection': 0}, 'endtime': {'time': '1200', 'timeSection': 0}}, 'offPeakPowerFlag': 0}, 'reserveChargeInfo2': {'reservChargeInfoDetail': {'reservInfo': {'day': [9], 'time': {'time': '1200', 'timeSection': 0}}, 'reservChargeSet': False, 'reservFatcSet': {'defrost': False, 'airTemp': {'value': '00H', 'unit': 0, 'hvacTempType': 1}, 'airCtrl': 0, 'heating1': 0}}}, 'reservFlag': 0, 'ect': {'start': {'day': 0, 'time': {'time': '0000', 'timeSection': 0}}, 'end': {'day': 0, 'time': {'time': '0000', 'timeSection': 0}}}, 'targetSOClist': [{'targetSOClevel': 80, 'dte': {'rangeByFuel': {'evModeRange': {'value': 299, 'unit': 1}, 'totalAvailableRange': {'value': 299, 'unit': 1}}, 'type': 2}, 'plugType': 0}, {'targetSOClevel': 100, 'dte': {'rangeByFuel': {'evModeRange': {'value': 384, 'unit': 1}, 'totalAvailableRange': {'value': 384, 'unit': 1}}, 'type': 2}, 'plugType': 1}]}, 'batteryPreconditioning': False, 'batterySoh': 0}, 'ign3': False, 'hoodOpen': False, 'transCond': True, 'steerWheelHeat': 0, 'sideBackWindowHeat': 0, 'tirePressureLamp': {'tirePressureLampAll': 0, 'tirePressureLampFL': 0, 'tirePressureLampFR': 0, 'tirePressureLampRL': 0, 'tirePressureLampRR': 0}, 'battery': {'batSoc': 81, 'batState': 0, 'sjbDeliveryMode': 0, 'batSignalReferenceValue': {'batWarning': 65}, 'powerAutoCutMode': 2}, 'lampWireStatus': {'stopLamp': {'leftLamp': False, 'rightLamp': False}, 'headLamp': {'headLampStatus': False, 'leftLowLamp': False, 'rightLowLamp': False, 'leftHighLamp': False, 'rightHighLamp': False, 'leftBifuncLamp': False, 'rightBifuncLamp': False}, 'turnSignalLamp': {'leftFrontLamp': False, 'rightFrontLamp': False, 'leftRearLamp': False, 'rightRearLamp': False}}, 'windowOpen': {'frontLeft': 0, 'frontRight': 0, 'backLeft': 0, 'backRight': 0}, 'smartKeyBatteryWarning': False, 'fuelLevel': 0, 'washerFluidStatus': False, 'breakOilStatus': False, 'sleepModeCheck': True, 'time': '20240923103001', 'remoteWaitingTimeAlert': {'remoteControlAvailable': 1, 'remoteControlWaitingTime': 168, 'elapsedTime': '00:03:08'}, 'systemCutOffAlert': 0, 'tailLampStatus': 0, 'hazardStatus': 0}, 'odometer': {'value': 1880, 'unit': 1}}}, 'msgId': '23134b50-79a0-11ef-9a43-bd5b0bcbc2cc'}

The specific lines with the data don't show timezones. My gut? They use some form of eventual consistency DB like nosql, this causes some data to be wrong from time to time. In theory you may catch it in the native app, maybe with a automation to tell you when this happens. If you can catch it you could report to Kia but not sold they will fix it.

Other option we can do is add logic that if the last updated goes back in time ignore the value. Any other datapoints we could drive this off of?

johny-mnemonic commented 1 month ago

@steps56 sorry, but then you completely missed what this reported issue is all about. We all know how to call force update to have the time in sync🙄 You unfortunately missed we are not talking about the time from background updates... Please, if you don't have an issue or don't see what is causing the bug (from the code or reported logs), do not post. Thank you.

@cdnninja so finally we have the root cause🎉 Nicely spotted! Could be that one of their servers has a bad config (i.e. bad TZ settings), so it is returning wrong data and causing it to be this random 🤔 I would vote for simple logic to update "last_updated_at" sensor only if the received time is higher than what is already stored in the sensor. I went through the debug logs and I don't see other place from which we could get the correct timestamp. We might find better way in the future but it will never make sense to update the sensor to go back in time, so it should be good enough, at least for now.

cdnninja commented 1 month ago

It could be that simple as well. Unfortunately hard for us to know!

I'll add some logic and we can give that a go.

fuatakgun commented 1 month ago

Instead of waiting @cdnninja to make a change, you can create a template sensor given that last_updated_at is part of attributes of debug sensor, right?

By the way, it is a very crucial sensor that helps us to decide when to enforce force update or stay with cached values, so great finding but weird issue.

Steven77nl commented 1 month ago

Thanks @fuatakgun for your workaround idea. I created a Input datetime entity that I update with the maximum date/time that that the last_update_at sensor gets populated with in the last 5 changes using Node-RED Not perfect, but for now I don't have a update time jumping up and down on my dashboard. Thx !

ZuinigeRijder commented 1 month ago

A possible workaround could be to add 2 (or timezone) hours (or convert from UTC to local timezone) when the latest last_updated_at is less than the previous last_updated at. But this needs to be done at the client side and it needs to know the previous last_updated_at.

cdnninja commented 1 month ago

@ZuinigeRijder this can be done in the library but it won't solve it for your monitor client based on my understanding on how it consumes the library.

Steven77nl commented 1 month ago

I used your idea @ZuinigeRijder to improve my temporary workaround. I got a date/time input helper to keep track of the most recent date/time and now use a function in Node-RED to update that helper when the received time is more recent (including those 2 extra hours, when the received time is less).

// Load global context for homeassitant
const ha = global.get("homeassistant")

// get the updated_at time received from the integration
var received = ha.homeAssistant.states["sensor.niro_last_updated_at"].state

// get the updated_at time stored in the date/time helper
var stored = ha.homeAssistant.states["input_datetime.niro_last_updated_at_max"].state

// convert to Unix TimeStamps
var recv_date = new Date(received)
var recv_uts = recv_date.getTime() / 1000
var stor_date = new Date(stored)
var stor_uts = stor_date.getTime() / 1000

// add two hours if last received is back in time
if (recv_uts < stor_uts) { recv_uts += 7200 }

// return additional values for debugging
msg.received = received
msg.stored = stored
msg.recv_date = recv_date
msg.recv_uts = recv_uts
msg.stor_date = stor_date
msg.stor_uts = stor_uts

// output to 1 if update is needed, output to 2 if update is not needed
if (recv_uts > stor_uts) {
  msg.update = true
  return [msg, null]
} else {
  msg.update = false
  return [null, msg]
}

image

Still not aways the correct time, but a lot more often than the previous workaround :)

ZuinigeRijder commented 1 month ago

@Steven77nl Problem is that you disregard older timestamp if I read your code correctly. But it is possible the data has been changed and you get still an older timestamp, which is not 2 hours before the previous one, but still older than the previous datetime......

I have a fix ready for hyundai_kia_connect_monitor, will make a fix release soon. Here is my python code (newest_updated_at and previous_updated_at are both datetime objects with timezone information):

            if newest_updated_at < previous_updated_at:  # new entry older than previous?
                utcoffset = newest_updated_at.utcoffset()
                newest_updated_at_corrected = newest_updated_at + utcoffset
                if (
                    newest_updated_at_corrected
                    >= previous_updated_at  # newest not too old?
                ):
                    newest_updated_at = newest_updated_at_corrected

There is still one situation, which cannot be solved. Assume previous timestamp was more than 2 hours ago. The new last_updated_at, is that the timestamp with the wrong 2 hours or is that the correct timestamp?

ZuinigeRijder commented 1 month ago

Workaround added in hyundai_kia_connect_monitor R3.26.0. HA or hyundai_kia_connect_api could maybe something similar, when the previous latest_updated_at is known.

Steven77nl commented 1 month ago

Hi @ZuinigeRijder, the date is also included in my workaround, so no issues there.

I agree that in that last situation you mention, there is no solution, and we will still end up with a time of two hours too early. Though I'm doing 15min interval cached data updates, so most of the time it corrects itself within the next 2 or 3 updates.

Forced updates I only do when there is a reason to do so, based on triggers that I can get from other sources, like:

This way I stay within the API limits, as I have the latest important status information always on my HA dashboard and in the original Kia Connect App.

Example, last night the Wallbox went into pause, because the car was full. So the automation request a force update to show on the dashboard the battery is 100%, that was at 4:50am. After this for me there is no need to get new information until I start driving my car later today.

image

cdnninja commented 1 month ago

Could someone edit the manifest file of the integration to 3.23.3 and reboot? Let me know how it goes.

ZuinigeRijder commented 1 month ago

@cdnninja As I understand the changes, you fixed this now only for ccs2 vehicles. My IONIQ 5 from 2021 has the same issue, but will not call _update_vehicle_properties_ccs2. So it will only partly fix the issue for HA users.

Also I do not understand the implementation in _update_vehicle_properties_ccs2. I think the previous code was also wrong, Date is not in UTC, so it should have been in the first place (also self.data_timezone in first part):

        if get_child_value(state, "Date"):
            # `Date` field is in UTC time
            vehicle.last_updated_at = parse_datetime(
                get_child_value(state, "Date"), self.data_timezone
            )
        else:
            vehicle.last_updated_at = dt.datetime.now(self.data_timezone)

And the new changes, the first check, should that be if vehicle.last_updated_at is NOT filled, e.g. like:

        if not vehicle.last_updated_at:

And I even expected an else in the new changes, see my solution here. I would expect a fix in the setter of vehicle.last_updated_at, with my solution.

cdnninja commented 1 month ago

Good catch on CCS2. Lets move this chat to a PR to fix it. Thank you for the help!

ZuinigeRijder commented 1 month ago

I can make a proposal Pull Request, but I cannot test for ccs2, but I can test for non-ccs2.

ZuinigeRijder commented 1 month ago

Lets hope that this workaround solves this issue and does not have side effects. Sure HA users will test this ;-)

cdnninja commented 1 month ago

This has been released in kia_uvo. Everyone please update to latest and let us know how it goes.

steps56 commented 1 month ago

https://github.com/user-attachments/assets/dc7c4100-a6e4-4858-a9cc-096bf39bfd30

This has been released in kia_uvo. Everyone please update to latest and let us know how it goes.

Working nice as before 👌. By the way, I never understood any "problems" about date/time mentioned here... Updated to newest .8 version, HA Restart and all working as usual... hitting "force update" plus "update" straight behind !!! (twice in video shown above, because I was to fast hitting the "update" after "force update") and it shows actual right day and time by minute 100% exactly. Perfekt by date/time as before 🤷🏼‍♂️

steps56 commented 1 month ago

Sloooow respons by the people which should have had 🤔 any "problems"... Anyway.... I still assume, if someone has date/time drifts, its not any root at the integration... If its working nice at my setup, its working nice basic. There may be a few possible "problems" with car/bluelink setup, or set time by HA or whatever. But seriously, its working very nice without any date/time shift.

Some artificial "problems" occure by let the car sit a few days unused, but that's wanted by basic Hyundai setup of bluelink to save the 12V Battery and easy disappears by driving next time. So, thanks for that great integration, giving many possibilities to use BlueLink by HA, Evcc, and, and ... 😎👌

artursulkowski commented 1 month ago

@cdnninja Just downloaded 2.26.8. First request after HA restart unfortunately returned time 2h in the past, another returned the correct one. As I understand - we won't avoid some messy API responses. But after the change, at least time should not be moving backwards - right? I kept kia_uvo logs on debug mode, will leave it for 24h and if I will spot any "time back travel" I will share some logs. Thanks for that workaround!

@steps56 problem is not artificial - e.g. I am using last_updated_at in my integration to know if SOC that I am reading is up to date or I need to ask for "fresh" one. With time jumping here and there, I could not estimate correctly SOC during the charging.

cdnninja commented 1 month ago

Yeah it would just not jump backward. Some bad data would still exist.

steps56 commented 1 month ago

@cdnninja Just downloaded 2.26.8. First request after HA @steps56 problem is not artificial - e.g. I am using last_updated_at in my integration to know if SOC that I am reading is up to date or I need to ask for "fresh" one. With time jumping here and there, I could not estimate correctly SOC during the charging.

Hello, I'm still not clear about any problem... as the use case about Electric-Car charging is normally/easy done like: 1. PV or whatever charging is taken by Evcc Addon. 2. Evcc "talks" to the Hyundai Ecar by its configuration. 3. Now as a may be fancy stuff, the discussed Hyundai Integration comes in, to deliver more or less fancy/interesting data... for dashboards or whatever, but useless for charging stuff.

Conclusion: THIS Integration is not necessary or even important for Ecar charging. Evcc (the Mercedes of Charging Software) will get all it's wanted more or less important data from ist configuration, including the Hyundai (or whatever) car data, like SOC and, and... In any point, the Hyundai (or Kia whatever named here discussed) integration is never part of the charging stuff. Evcc Addon communicates always without this integration to the cars BlueLink data. Example: We have been at a short 6 days vacation till yesterday. The Hyundai Ioniq5's BlueLink doesn't respond to updating, as the car wasn't used some days, programmed exactly that way by Hyundai, to save it's 12 V Battery System. So no way to update by BlueLink App and so no way by this integration for sure. AFTER driving the car a few minutes, its Systems are up to date with the BlueLink Hyundai Server and again usable by this integration (force update working again). Same with Evcc... no SOC available (workaround in this care: use guest car), as no BlueLink up to date data by off line communikation as a battery saver deep sleep of this system. So its all good and as wanted by Hyundai. Again: This integration is neither usable nor important for Hyundai charging by any mameworthy Wallbox. It's all just about BlueLink connection established and good Evcc configuration. This intergration diecussed here, is "just" nice to get all the fancy data from the car, for setting up of more or less nice "control" dashboards and whatever trigger stuff 😉.

Don't understand me wrong, I love this integration... but it's nothing to be really usable about Ecar charging. Don't try to reinvent the wheel... Evcc does everything perfekt, no need to charge by using/setting up fancy wannebee Charging solution by homemade stuff using this integration 🤣.... and so no really need for exact day/time... but the always was exact day/time too at my setup 😉... despite others usecase 🤷🏼‍♂️.

cdnninja commented 1 month ago

@steps56 are you referencing evcc from GitHub? I'm glad that works for you. It's a different purpose and also "home grown". This feels like you are trying to insult the work done on this.

I'm not sure what your goal is here. An issue did exist that impacted some people's use case. Not caused by our code but corrected in it.

The data proves that. I would appreciate you stop trying to create drama.

johny-mnemonic commented 1 month ago

@steps56 I am confused why you are spamming this conversation by useless posts even containing images and video, to boast about your nice HA dashboard for your car. Is it really that hard to understand that if you don't have the reported issue nor anything to bring to fix the issue for others, it is pointless to post here? 😮 FYI: Not everyone is using EVCC. It doesn't support my charger for example.

@cdnninja thanks for the fix! ❤️

@artursulkowski yep, same here. I also use this sensor to know whether to ask for the update to have up-to-date SOC in HA as I am handling car charging by my own automation.

cdnninja commented 1 month ago

Credit goes to @ZuinigeRijder

artursulkowski commented 1 month ago

After running 2.26.8. for two days I can confirm that workaround works: last_updated_at does not move backwards. Still (as expected) the issue of inconsistent last_updated_at times exists but it is rather EU server issue. Hopefully this will be fixed at some point by Hyundai/Kia.