ISSUES WITH BETA RELEASE

robinostlund commented 10 months ago

Only report issues here with the latest beta release and do not start any discussion here!

Thanks

oliverrahner commented 10 months ago

@oliverrahner No Problem, without you and the other guy's effort I wouldn't have any information inside HA. :-) So better do a bunch of try-and-error and we'll get it reliable again.

Do you want me after midnight to keep the 4.4.68 running or should it be disabled until you had the time to make changes.?

@gohakn It's really up to you, if you do not mind potentially being locked out of the app until the next day, just keep using it. The next days, I won't have much time. I hope to get the control stuff done today, but won't be able to work on the request reduction until probably next Monday.

oliverrahner commented 10 months ago

@sebking1971 Any log entries?

oliverrahner commented 10 months ago

I also tested version 4.4.68 and it seems to work for my Tiguan E-Hybrid. There is one thing I am missing, but this was also not available in the old version: I would like to be able to set the maximum charging current from Homeassistant. Currently this is just a readonly value. The reason why I would like to have this feature: I want to control the charging current dependent on the status of my photovoltaic installation via an automation in Homeassistant. Is it possible to enable this value as read/write?

@squonk11 I can check this next week, if it is settable via the app, I should be able to include it in the lib.

oliverrahner commented 10 months ago

@enoch85 @shift-del1 @florie1706 This all seems to be related to the apparently too high amount of requests we are sending. Reducing the poll interval as mentioned above should remove this issue until we found a better way.

florie1706 commented 10 months ago

@enoch85 @shift-del1 @florie1706 This all seems to be related to the apparently too high amount of requests we are sending. Reducing the poll interval as mentioned above should remove this issue until we found a better way.

Indeed it is working again today morning, just changed the poll interval form 5 minutes to 10 minutes. Let's see how it works now ;)

virtualdj commented 10 months ago

@squonk11 I can check this next week, if it is settable via the app, I should be able to include it in the lib.

@oliverrahner Just check that the service is working, like in the picture I've posted above.

squonk11 commented 10 months ago

I also tested version 4.4.68 and it seems to work for my Tiguan E-Hybrid. There is one thing I am missing, but this was also not available in the old version: I would like to be able to set the maximum charging current from Homeassistant. Currently this is just a readonly value. The reason why I would like to have this feature: I want to control the charging current dependent on the status of my photovoltaic installation via an automation in Homeassistant. Is it possible to enable this value as read/write?

@squonk11 I can check this next week, if it is settable via the app, I should be able to include it in the lib.

@oliverrahner : It was settable with the previous version of the app (Volkswagen WeConnect) but with the new app (Volkswagen App) it is not settable any more - they removed this important feature from the app.

squonk11 commented 10 months ago

@squonk11 In your automation, use the action Call a service and set it up like below.

Using hybrid vehicles you can only set it to max or reduced.

@virtualdj : Thank you, I will try this during the next days.

oliverrahner commented 10 months ago

@squonk11 I can check this next week, if it is settable via the app, I should be able to include it in the lib.

@oliverrahner Just check that the service is working, like in the picture I've posted above.

@virtualdj But if it isn't possible to set anymore in the app, I can't make that service work again... The backend has been completely reworked by VW, so we can't even check (easily) if they only removed it from the frontend, but it's still there as a feature on their servers.

artspb commented 10 months ago

Thanks so much for your work! Very much appreciate it.

The following sensors are missing now. Is it known/expected? I really use only the first one, though. The rest won't be missed.

Charging time left
Heater source for departure timers
Last connected
Minimum charge level for departure timers
Parking time

Another thing is that there are now buttons for things that my Passat GTE 2018 doesn't support, such as locking doors/trunk. But I'd assume that as this part isn't fully finished yet, they'll lose their ability to change the state at a later point. Not a big deal anyway.

Nevertheless, thanks again for bringing the integration back to life!

zap-o-post commented 10 months ago

First of all I would like to thank @robinostlund and @oliverrahner for their great work. Version 4.4.68 is running so far. However, all of my entity IDs have now changed, which is causing problems with automations.

e.g. before: tiguan_carid_battery_level now: tiguan_longname_carid_battery_level

Where can I change this entity name?

milkboy commented 10 months ago

First of all I would like to thank @robinostlund and @oliverrahner for their great work. Version 4.4.68 is running so far. However, all of my entity IDs have now changed, which is causing problems with automations.

e.g. before: tiguan_carid_battery_level now: tiguan_longname_carid_battery_level

Where can I change this entity name?

That would be the "name" when you add the integration (defaults to VIN)

It might be possible to rename it to fix the entities, but if not, then remove the integration and add it back

gohakn commented 10 months ago

@oliverrahner Today at midnight ( when the counter at VW was reset) and the Integration started working again I changed the Update interval from 5 to 10 Minutes but now at 11:50 I'm locked out again.

Timer2000 commented 10 months ago

Is possible that it is not an Intervall Problem? I have an Iobroker Adapter for VW, that has an Intervall with 8min. And It was no Problem. In the Konfig can Set an Force Request and much more. I try it without them

virtualdj commented 10 months ago

But if it isn't possible to set anymore in the app, I can't make that service work again...

@oliverrahner @squonk11 It is available on the new Volkswagen app and has been there since the beginning. It's under Settings > Reduced AC current and it's a switch.

Reduced AC setting (Italian)

This feature is also working in the other backend library if you need to get the URL.

What was missing initially in the Volkswagen app was the windshield heather, but it has been added now.

SkinnyGnome commented 10 months ago

Hi all,

First of all, thanks for the great work getting this to work again. 👍🏼

I got the Beta (v4.4.68) to work for a while, but after some time all sensors where unavailable i HomeAssistent. I then checked The Volkswagen app, and all sensors has disappeared from there also? Same, when I check the WeConnect website.

Is this the API thats been blocked? if so how do I get it unblocked again?

Thanks

SkinnyGnome commented 10 months ago

Hi all,

First of all, thanks for the great work getting this to work again. 👍🏼

I got the Beta (v4.4.68) to work for a while, but after some time all sensors where unavailable i HomeAssistent. I then checked The Volkswagen app, and all sensors has disappeared from there also? Same, when I check the WeConnect website.

Is this the API thats been blocked? if so how do I get it unblocked again?

Thanks

Ah, it could be the same as this: https://github.com/robinostlund/homeassistant-volkswagencarnet/issues/507#issuecomment-1832792232

I will try to remove it from Homeassistsnt and wait until tomorrow 🤔

Jaspils commented 10 months ago

To begin, I really appreciate all work on this integration. I've been using it for over a year now and preferred using this over the Native Volkswagen app. Since they moved API to this new app, it has been a pain to keep track with my car. I'm very glad this new version brings it back to life!

For now my main issue is the missing features to control my parking heater (especially in the cold days now). Below the information on all sensors that my car (Arteon 2017 Diesel with Auxilary heater) had before, and which were recovered with the latest update:

Here some debug logging that shows its unsupported, while it is a feature that was available before, and still available in the new Volkswagen App (so at least the car supports is)

2023-11-30 14:15:06.386 DEBUG (MainThread) [volkswagencarnet.vw_dashboard] WVWZZZ** Auxiliary Climatisation (AuxiliaryClimatisation:auxiliary_climatisation) is not supported 2023-11-30 14:15:06.395 DEBUG (MainThread) [volkswagencarnet.vw_dashboard] WVWZZZ** Parking Heater heating/ventilation status (Sensor:pheater_status) is not supported 2023-11-30 14:15:06.395 DEBUG (MainThread) [volkswagencarnet.vw_dashboard] WVWZZZ** Parking Heater heating/ventilation duration (Sensor:pheater_duration) is not supported

There is some more, but the post is long enough as it is now. I hope this helps with development.

Driekes commented 10 months ago

Again 429 issues even after setting refresh to 10 min.

Also I reloaded the integration and now almost all entities have gone. In the log file there is a lot of mention of: Platform volkswagencarnet does not generate unique IDs

Keep up the good work! I also emailed volkswagen they should move from 1990 and vendor lock in to 2023 connected mindset :)

enoch85 commented 10 months ago

Again 429 issues even after setting refresh to 10 min.

Also I reloaded the integration and now almost all entities have gone. In the log file there is a lot of mention of: Platform volkswagencarnet does not generate unique IDs

Keep up the good work! I also emailed volkswagen they should move from 1990 and vendor lock in to 2023 connected mindset :)

I've changed to 15 minutes. Will see tonight what happens during tomorrow.

mrandt commented 10 months ago

Using v4.4.68 (beta) I could add a Passat GTE and e-Golf (both 2020 models). Login worked without problems.

General info seems to be OK.

Control does not work. Do you need stacktraces for electric window heaters or electric climatization? Please let me know, I'll happily provide logs or collect any other info that may help.

Thank you very much for your work on this!

m47een commented 10 months ago

Firstly thanks for your hard work on this, I really appreciate it.

I've only recently bought our e-Golf and it's our first electric car too. I live In the UK and we have an energy supplier called Octopus Energy which has favourable tariffs for electric car owners. I've found that there is an issue with the Octopus Energy integration if I use this at the same time. I experience what others are referring to the API lockout. I don't have the full details of how the Octopus Energy integration with VW Connect works so I can't say how often the API is invoked on the HA side I set the update interval to 10 minutes and still had issues.

Happy to provide logs if someone could please offer some guidance on where to find them.

Thanks

squonk11 commented 10 months ago

But if it isn't possible to set anymore in the app, I can't make that service work again...

@oliverrahner @squonk11 It is available on the new Volkswagen app and has been there since the beginning. It's under Settings > Reduced AC current and it's a switch.

This feature is also working in the other backend library if you need to get the URL.

What was missing initially in the Volkswagen app was the windshield heather, but it has been added now.

@virtualdj : In the settings of my Volkswagen App there is no setting for the max. charging current. tempFileForShare_20231201-094033

Harted commented 10 months ago

Hi,

About the rate limiting problem.

I did some testing myself and when I got 429 too many request and kept polling I eventually got 502 Bad Gateway. My volkswagen app stopped working and it took about 24 hours to get it back.

I use(d) https://requestly.io/ as a proxy between my iPhone and can see the requests the volkswagen app makes. It updates the status every 60 seconds. The vehicle position it only fetches once when opening the app and so on...

You can also see the requests it makes to log in, commands and settings. It helped me a lot and hope it can help you too. I'm not really experienced with python so.. this is where my input stops 😆

thomasblankschein commented 10 months ago

Hi @robinostlund and @oliverrahner, thanks for your great work! I bought you a coffee each :-)

Using v.4.4.68 with a VW eUp 2020, I noticed the following issues:

1. Last connected doesn't exist any more => not available via the new API?

2. Calculation of total electric consumption I used to calculate the total electric consumption by this template:

    eup_last_trip_total_electric_consumption:
      value_template: >-
        {{ (   states('sensor.eup_last_trip_average_electric_engine_consumption') | float
             + states('sensor.eup_last_trip_average_auxillary_consumer_consumption') | float
             - states('sensor.eup_last_trip_average_recuperation') | float
           ) | round(1) }}
      unit_of_measurement: 'kWh/100 km'

sensor.eup_last_trip_average_electric_engine_consumption => OK

sensor.eup_last_trip_average_auxillary_consumer_consumption => Not available

sensor.eup_last_trip_average_recuperation => Is available, but with factor 0,1 compared to the value in the app => Bug?

virtualdj commented 10 months ago

@squonk11 Interesting and terrible at the same time! Both you and me have an hybrid plug-in vehicle and I can assure you that setting works in my setup.

oliverrahner commented 10 months ago

@mrandt i have already collected some API calls for controls, just need to implement them, unfortunately not before Monday @Harted I didn’t know there was a cloud service for this, I use Fiddler locally. Then again, I guess that was to be expected 😂 I also saw these 429/502s, but no information about remaining limits or something like this. @thomasblankschein last updated should be possible to implement, will look at this next week. In my test cases I didn’t see the information regarding recuperation and aux consumption, but now I remembered that I only used the hybrid car information provided by @michiii1337, not the electric car information from @TheBlackBGD. I will check this again against information of his account.

If anybody wants to help, it would be great to retrieve logs from requestly.io, as mentioned by @Harted above, while viewing/controlling in the app what you want implemented. You could send those logs to <firstname>@<lastname>.me

oliverrahner commented 10 months ago

As an additional note: what would be very interesting to see would be a situation where the app is opened and any information from the car is updated externally, like plug connected/disconnected, ignition/movement, lock/unlock. That could help in evaluating if/how we could optimize data retrieval.

Harted commented 10 months ago

@oliverrahner you've got mail.

About updates from the car: I tested locking/unlocking.. It updates after about 1 minute. I'm doing status update requests with this url:

https://emea.bff.cariad.digital/vehicle/v1/vehicles//selectivestatus?jobs=access,charging,climatisation,fuelStatus

I took out departureTimers, honkAndFlash, userCapabilities & vehicleLights from the jobs query variable because these things don't really interest me. I can poll every 20 seconds without any error.. (While I was writing this I've got a too many requests 🤣, so forget about the 20 seconds) ..but because the api returns the updated value from the car only after a minute I don't think it's needed to do so many poll requests.

So, everything you see updated in the app is with these one minute polls the app does. Even when you do a command, it is waiting for the next update to check if the command is completed successfully, has failed, or is pending.

When the battery is too low and you ask to start climatisation on battery for example, you see the climatisation is active, only to see after about a minute, the command has failed because the battery is too low.

enoch85 commented 10 months ago

Changed polls to 59 minutes yesterday and it's still working. Will report back how long it lasts.

virtualdj commented 10 months ago

When you ask to start climatisation on battery for example, you see the climatisation is active, only to see after about a minute, the command has failed because the battery is too low

I saw the same and sometimes also the climatisation started but the app displayed a message with "something didn't work / try again later"... Very very unrealiable on their side.

@oliverrahner What do you think about adding an entity switch that enables/disables the polling/updating of the data, at least until the subscription update mode works?

Maybe someone is interested in having frequent updates but not during the whole day... Having a switch (that can be further controlled with an automation in HA or handled manually) could be a possible workaround to the 429 errors.

The switch should only bypass the coordinator update, so if it's switched off (preventing polling) eventual controls like charging/discharging or manual updates (side question: does calling the service homeassistant.update_entity work with this integration?) should still update the entities, but in a "one shot".

enoch85 commented 10 months ago

When you ask to start climatisation on battery for example, you see the climatisation is active, only to see after about a minute, the command has failed because the battery is too low

I saw the same and sometimes also the climatisation started but the app displayed a message with "something didn't work / try again later"... Very very unrealiable on their side.

@oliverrahner What do you think about adding an entity switch that enables/disables the polling/updating of the data, at least until the subscription update mode works?

Maybe someone is interested in having frequent updates but not during the whole day... Having a switch (that can be further controlled with an automation in HA or handled manually) could be a possible workaround to the 429 errors.

The switch should only bypass the coordinator update, so if it's switched off (preventing polling) eventual controls like charging/discharging or manual updates (side question: does calling the service homeassistant.update_entity work with this integration?) should still update the entities, but in a "one shot".

This is a great idea! I only need values between 21:00 and 05:00 when the cars are charging. Doing a poll at 20:59 and then every hour until 05:00 would be perfect for me.

janchlebek commented 10 months ago

Maybe someone is interested in having frequent updates but not during the whole day... Having a switch (that can be further controlled with an automation in HA or handled manually) could be a possible workaround to the 429 errors.

That is really a great idea! I only need data (and eventually controls) during certain times in a day. 👍👍

Harted commented 10 months ago

Maybe someone is interested in having frequent updates but not during the whole day... Having a switch (that can be further controlled with an automation in HA or handled manually) could be a possible workaround to the 429 errors.

Yes, very good idea. I think I've got too many errors yesterday because I opened the app a couple times while my polling was running. So either advise the users not to use the volkswagen app, or like @virtualdj advised, a switch to disable or limit polling.

thomasblankschein commented 10 months ago

Maybe someone is interested in having frequent updates but not during the whole day... Having a switch (that can be further controlled with an automation in HA or handled manually) could be a possible workaround to the 429 errors.

Thumbs up from me! During night I don't need frequent updates. But during charging an update every x minutes would be nice.

Sikk87 commented 10 months ago

(side question: does calling the service homeassistant.update_entity work with this integration?) should still update the entities, but in a "one shot".

update_entity did not work in my case. reload_config_entry did the trick.

virtualdj commented 10 months ago

If anybody wants to help, it would be great to retrieve logs from requestly.io, as mentioned by @Harted above, while viewing/controlling in the app what you want implemented.

@Harted I believe you're on iOS, right?

Because on Android when applying the proxy to the Wi-Fi connection it's only able to capture the browser traffic, while the Volkswagen app if already logged in doesn't connect (with a red error message) and if I logout and try to login again a white/blank page appears (instead of asking for my credentials).

Harted commented 10 months ago

@virtualdj yes I am on iOS.

enoch85 commented 10 months ago

Changed polls to 59 minutes yesterday and it's still working. Will report back how long it lasts.

I can now confirm that this works! It's been online for 2 days now, and no missing enteties.

FYI: I've avoided to check the Volkswagen app as well.

oliverrahner commented 10 months ago

For now my main issue is the missing features to control my parking heater (especially in the cold days now).

@Jaspils I didn't have access to a car with "legacy" parking heating. Would you be able to provide a data dump (as described by @Harted above) or would be willing to pass me your credentials for a certain time? Then I should pretty easily be able to work this out.

oliverrahner commented 10 months ago

@virtualdj

Because on Android when applying the proxy to the Wi-Fi connection it's only able to capture the browser traffic, while the Volkswagen app if already logged in doesn't connect (with a red error message) and if I logout and try to login again a white/blank page appears (instead of asking for my credentials).

Maybe the Android app uses certificate pinning? But then, I'm not sure why they wouldn't do the same for iOS. There is a method to remove certificate pinning requirements from an Android app on a rooted device using adb, but I won't go into details here 😬

michiii1337 commented 10 months ago

For now my main issue is the missing features to control my parking heater (especially in the cold days now).

@Jaspils I didn't have access to a car with "legacy" parking heating. Would you be able to provide a data dump (as described by @Harted above) or would be willing to pass me your credentials for a certain time? Then I should pretty easily be able to work this out.

what do you mean with "leagacy" parking heating? my car has a parking heater from vw with petrol. and I have the option of pre-conditioning via the hybrid battery

oliverrahner commented 10 months ago

@michiii1337 I got the impression (for example from this code line) that the "parking heater" is the legacy version of the "climatisation", that newer cars such as yours have. So I would assume that newer cars only use the climatisation API, older cars (if they have a parking heater) only use the "parking heater" APIs. Do you have other insight?

Jaspils commented 10 months ago

For now my main issue is the missing features to control my parking heater (especially in the cold days now).

@Jaspils I didn't have access to a car with "legacy" parking heating. Would you be able to provide a data dump (as described by @Harted above) or would be willing to pass me your credentials for a certain time? Then I should pretty easily be able to work this out.

I'm not entirely sure how to intercept and gather the logs as @Harted mentioned. Perhaps your second proposal is easier to manage for us both. Assuming you don't missuse it and randomly honk my car 😆

@oliverrahner What is the safest way to manage this? Volkswagen insist they have a feature to share a car to multiple users, but I can't find any button of function to actually invite a new user. I've send you an e-mail to discuss this further in a more private way.

virtualdj commented 10 months ago

Maybe the Android app uses certificate pinning? But then, I'm not sure why they wouldn't do the same for iOS.

@oliverrahner No, it doesn't use certificate pinning. It's an unfortunate "feature" of Android, that's missing on iOS, that prevents installing system-wide CA certificates on non-rooted phones (see here). So if you want an user certificate to be uses by apps you must be rooted or the app must opt-in for that (and Volkswagen app obviously does not).

virtualdj commented 10 months ago

Volkswagen insist they have a feature to share a car to multiple users, but I can't find any button of function to actually invite a new user.

@Jaspils If I remember well on the old We Connect app there was a button to invite users, so I'm pretty sure they refer to that feature (because remember: they aren't aware of these changes!). In the current Volkswagen app we have Settings > Administration > Manage users but there's not a button to create or invite new users.

danielszilagyi commented 10 months ago

@oliverrahner No, it doesn't use certificate pinning. It's an unfortunate "feature" of Android, that's missing on iOS, that prevents installing system-wide CA certificates on non-rooted phones by apps you must be rooted or the app must opt-in for that (and Volkswagen app obviously does not).

I thought the same, but then gave it a shot and could create and use an apk based on the VW app with apk-mitm and intercept the traffic between the app and the api with pcapdroid + its mitm proxy.

I shared my results with Robin back then, since I’m nowhere in python. Drop me a mail if the apk can be interesting to anyone. It worked on an unrooted phone with the pcapdroid mitm ca installed. My gh handle as left with gmail on the right part works.

janrothkegel commented 10 months ago

Also thanks from my side for the developer's hard work :-)

I just wonder a little bit because my battery level according to the app has been 55% for 35 minutes but the integration does not update. Update interval is set to 30 minutes.

reloading the integration did not help
updateentity did not work (as described earlier in this thread) **--> Not correct: It actually works, but only if the data are synchronized via App before._**
reload_config_entry -> here I don't know what to put into config entry ID field

Any ideas are very welcome

grafik

gohakn commented 10 months ago

@janrothkegel The same with my E-UP since the last 3 days. Only if i shutdown the Integration and start it again it`s getting the actual data.

larsmarius87 commented 10 months ago

Strange problem, when i use this integration, all of the entites is showing as unavailble, and i get 429 error in log. VW app works, and another third party app, Tibber (norwegian electric company) also seems to be working. And if i try the "custom" integaration made by @mitch-dc it works to log in and i do get all the entities updated.

edit; 4 entities seems to update in this integration: Force data refresh, Request in progress, Request results, Requests remaining

2nd edit; it works again, had to delete and add the car altogheter..

Hope this info can be usefull :)

robinostlund / homeassistant-volkswagencarnet

ISSUES WITH BETA RELEASE #511

Only report issues here with the latest beta release and do not start any discussion here!

Thanks