home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.31k stars 30.62k forks source link

Renault integration not working - ClientResponseError: 403, message='Forbidden' #91593

Closed mathieucarbou closed 1 year ago

mathieucarbou commented 1 year ago

The problem

Renault integration cannot connect anymore to Renault servers.

This happend after upgrading to core-2023.4.5

Logger: homeassistant.config_entries
Source: components/renault/renault_hub.py:59 
First occurred: 10:56:08 (1 occurrences) 
Last logged: 10:56:08

Error setting up entry Renault for renault
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 383, in async_setup
    result = await component.async_setup_entry(hass, self)
  File "/usr/src/homeassistant/homeassistant/components/renault/__init__.py", line 29, in async_setup_entry
    await renault_hub.async_initialise(config_entry)
  File "/usr/src/homeassistant/homeassistant/components/renault/renault_hub.py", line 59, in async_initialise
    vehicles = await self._account.get_vehicles()
  File "/usr/local/lib/python3.10/site-packages/renault_api/renault_account.py", line 62, in get_vehicles
    return await self.session.get_account_vehicles(
  File "/usr/local/lib/python3.10/site-packages/renault_api/renault_session.py", line 188, in get_account_vehicles
    return await kamereon.get_account_vehicles(
  File "/usr/local/lib/python3.10/site-packages/renault_api/kamereon/__init__.py", line 239, in get_account_vehicles
    await request(
  File "/usr/local/lib/python3.10/site-packages/renault_api/kamereon/__init__.py", line 160, in request
    http_response.raise_for_status()
  File "/usr/local/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 1005, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 403, message='Forbidden', url=URL('https://api-wired-prod-1-euw1.wrd-aws.com/commerce/v1/accounts/<uuid>/vehicles?country=FR')
Logger: homeassistant.components.renault.renault_vehicle
Source: helpers/update_coordinator.py:193 
Integration: Renault (documentation, issues) 
First occurred: 10:53:28 (2 occurrences) 
Last logged: 10:53:28

Error requesting <vin> cockpit data: 403, message='Forbidden', url=URL('https://api-wired-prod-1-euw1.wrd-aws.com/commerce/v1/accounts/<uuid>//kamereon/kca/car-adapter/v2/cars/<vin>/cockpit?country=FR')
Error requesting <vin> location data: 403, message='Forbidden', url=URL('https://api-wired-prod-1-euw1.wrd-aws.com/commerce/v1/accounts/<uuid>//kamereon/kca/car-adapter/v1/cars/<vin>/location?country=FR')

What version of Home Assistant Core has the issue?

core-2023.4.5

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Supervised

Integration causing the issue

renault

Link to integration documentation on our website

https://www.home-assistant.io/integrations/renault/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 1 year ago

Hey there @epenet, mind taking a look at this issue as it has been labeled with an integration (renault) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `renault` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign renault` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


renault documentation renault source (message by IssueLinks)

epenet commented 1 year ago

It looks like a change on the Renault servers - not linked to 2023.4.5 We might need a new API key, similar to https://github.com/hacf-fr/renault-api/pull/552

Please open an issue on renault-api repository.

mathieucarbou commented 1 year ago

@epenet : from where these API key come from ? Is it something that can be made configurable in HA ? Could we generate some ?

OPENED https://github.com/hacf-fr/renault-api/issues/847

epenet commented 1 year ago
mathieucarbou commented 1 year ago

Oh crap! If I understand correctly, the only way to get this api key is by intercepting it from https calls by decompiling the renault android app ?

Or is there an easier way like described here ? https://github.com/jamesremuscat/pyze/issues/113

Note: making it configurable would help a lot because users running with HA OS cannot modify the constant file to change the api key since this is a core integration.

Here is what I get at https://renault-wrd-prod-1-euw1-myrapp-one.s3-eu-west-1.amazonaws.com/configuration/android/config_en_GB.json

    "servers": {
        "kamareon": {
            "target": "https://alliance-platform-serviceadapter-staging.apps.prod.eu.kamereon.org",
            "apikey": "Z7P4xyNrTITzNZh52oObSOVfKns9XGLE"
        },
        "wired": {
            "target": "https://api-wired-dev-1-euw1.wrd-aws.com",
            "apikey": "AHdOWFASWEPUVQVlhJWshsios0FqTG2E"
        },
        "wiredValid": {
            "target": "https://api-wired-valid-1-euw1.wrd-aws.com",
            "apikey": "AHdOWFASWEPUVQVlhJWshsios0FqTG2E"
        },
        "wiredProd": {
            "target": "https://api-wired-prod-1-euw1.wrd-aws.com",
            "apikey": "oF09WnKqvBDcrQzcW1rJNpjIuy7KdGaB"
        },
        "gigya": {
            "target": "https://accounts.eu1.gigya.com",
            "apikey": "3_S0OWIrqeJ6mxOkXFT8i3TTDwW1IGKk2rIypZjGXi4Hh8vce2ohuERio1Ka5DBbUr"
        },
        "gigyaValid": {
            "target": "https://accounts.eu1.gigya.com",
            "apikey": "3_ZId52TSAYEDjI2qr65kV8M2PM2awtpGrNKqAe6zPNeWxlbvNFNSMlrQMC78fgUWb"
        },
        "gigyaProd": {
            "target": "https://accounts.eu1.gigya.com",
            "apikey": "3_e8d4g4SE_Fo8ahyHwwP7ohLGZ79HKNN2T8NjQqoNnk6Epj6ilyYwKdHUyCw3wuxz"
        }
    },
epenet commented 1 year ago

The AWS url is the original keys, before they were hidden away.

TheReiner commented 1 year ago

i try a thing... i use Postman, enable Proxy... in my iPhone (with Proxy), i have open MyRenault. And this the result (With a google API)

Maybe it will help you ?

Screenshot 2023-04-18 at 14 44 12

epenet commented 1 year ago

Thanks - proxy is block on Android.

The new key is YjkKtHmGfaceeuExUDKGxrLZGGvtVS0J Do you want to do the honours and update the KAMEREON_APIKEY in the constant file on renault-api repository? https://github.com/hacf-fr/renault-api/blob/d838c59b823ed25a7a6ce7c0bdd2bc7ff993fd30/src/renault_api/const.py#L21

Jarige commented 1 year ago

May I compliment you guys with the fast response to this? Thanks! Sounds like I won't have to wait too long for it to be fixed.

TheReiner commented 1 year ago

Thanks - proxy is block on Android.

The new key is YjkKtHmGfaceeuExUDKGxrLZGGvtVS0J Do you want to do the honours and update the KAMEREON_APIKEY in the constant file on renault-api repository? https://github.com/hacf-fr/renault-api/blob/d838c59b823ed25a7a6ce7c0bdd2bc7ff993fd30/src/renault_api/const.py#L21

For Android, you need ton install Postman certificate ^^ like iOS ^^ https://learning.postman.com/docs/sending-requests/capturing-request-data/capturing-https-traffic/#android

mathieucarbou commented 1 year ago

Thanks - proxy is block on Android.

The new key is YjkKtHmGfaceeuExUDKGxrLZGGvtVS0J Do you want to do the honours and update the KAMEREON_APIKEY in the constant file on renault-api repository? https://github.com/hacf-fr/renault-api/blob/d838c59b823ed25a7a6ce7c0bdd2bc7ff993fd30/src/renault_api/const.py#L21

For those with HA OS, do we have to wait for a new HA release ?

epenet commented 1 year ago

Step 1 => create a PR on renault-api to update the constant and then wait for merge + new release Step 2 => create a PR on HA with the updated renault-api package and then wait for merge + new HA patch release

mathieucarbou commented 1 year ago

Do you want to do the honours and update the KAMEREON_APIKEY in the constant file on renault-api repository? https://github.com/hacf-fr/renault-api/blob/d838c59b823ed25a7a6ce7c0bdd2bc7ff993fd30/src/renault_api/const.py#L21

Small PRs like that can be done directly online. I did it: https://github.com/hacf-fr/renault-api/pull/848

mathieucarbou commented 1 year ago

For Android, you need ton install Postman certificate ^^ like iOS ^^ https://learning.postman.com/docs/sending-requests/capturing-request-data/capturing-https-traffic/#android

@TheReiner : I've tried a while ago and the manifest of the renault app does not allow to trust all certificates. So this step from the doc cannot be achieved:

Need to capture requests from an Android app? To capture requests from an Android app, you need to add a network security configuration file to your app to trust the postman-proxy-ca.crt certificate. For more information, see Trust additional CAs on the Android Developers portal.

JeedHome44 commented 1 year ago

Hello, Same problem here. I see you found a solution. Can we do manually correct it or we have to wait new update HA ? Thank you

Lantoit commented 1 year ago

Same problem here (it_IT)

epenet commented 1 year ago

PR created https://github.com/home-assistant/core/pull/91609 Need to wait for merge and for the next patch release.

In the mean time, you can either:

If you are extra motivated, you can create a PR to add an option flow so that next year the key can be set manually without waiting for a review process.

mathieucarbou commented 1 year ago

@TheReiner : I've installed Postman + the certificates on both my Android phone and iPad (IOS) but both didn't work: I was unable to intercept any ssl traffic. Maybe because of the new SSL TLS v3 spec which does not allow content renegotiation ? I would be glad to know how you make that work...

JeedHome44 commented 1 year ago
  1. I extracted the renault folder from the core.
  2. i put the folder in custom_component folder.
  3. I replaced the renault-api version in 0.1.13 in the manifest.
  4. I removed the renault integration
  5. i reload home assistant. but when i search for renault in the integrations, i don't see the carton logo. normally, home assistant prioritizes the custom folder if it has the same name, right? can someone tell me what i did wrong?
smgt commented 1 year ago

I run HA in a docker container (homeassistant/home-assistant sha256:bd5b10c7e4ae2f87380c1a4da2f211864c2b3b93c03c14ec132d838486df490e) and would like this fixed before the next release.

So this is what I did. I entered the container with docker exec -it home-assistant bash. Then entered the directory for the renault api package, cd /usr/local/lib/python3.10/site-packages/renault_api. There is the file, const.py, which I edited with vi (vi const.py) and replaced the api key mentioned in this thread (YjkKtHmGfaceeuExUDKGxrLZGGvtVS0J). After a HA restart it all worked again.

epenet commented 1 year ago

I run HA in a docker container and would like this fixed before the next release.

So this is what I did. I entered the container with docker exec -it home-assistant bash. Then entered the directory for the renault api package, cd /usr/local/lib/python3.10/site-packages/renault_api. There is the file, const.py, which I edited with vi (vi const.py) and replaced the api key mentioned in this thread (YjkKtHmGfaceeuExUDKGxrLZGGvtVS0J). After a HA restart it all worked again.

You shouldn't do this. You should instead make the same update as my PR: update the manifest file to use latest renault-api package version.

JeedHome44 commented 1 year ago

@epenet when i reload HA i don't see carton logo in Integrations ? how please ?

smgt commented 1 year ago

I run HA in a docker container and would like this fixed before the next release. So this is what I did. I entered the container with docker exec -it home-assistant bash. Then entered the directory for the renault api package, cd /usr/local/lib/python3.10/site-packages/renault_api. There is the file, const.py, which I edited with vi (vi const.py) and replaced the api key mentioned in this thread (YjkKtHmGfaceeuExUDKGxrLZGGvtVS0J). After a HA restart it all worked again.

You shouldn't do this. You should instead make the same update as my PR: update the manifest file to use latest renault-api package version.

This was the easiest for me without the need to learn anything. Just read the error message, copy the key from this thread and then paste it in a file. I doesn't really matter anyway since the next time I update the container the change will be gone.

Wendelstein7 commented 1 year ago

I run HA in a docker container (homeassistant/home-assistant sha256:bd5b10c7e4ae2f87380c1a4da2f211864c2b3b93c03c14ec132d838486df490e) and would like this fixed before the next release.

So this is what I did. I entered the container with docker exec -it home-assistant bash. Then entered the directory for the renault api package, cd /usr/local/lib/python3.10/site-packages/renault_api. There is the file, const.py, which I edited with vi (vi const.py) and replaced the api key mentioned in this thread (YjkKtHmGfaceeuExUDKGxrLZGGvtVS0J). After a HA restart it all worked again.

This worked like a charm. Thank you for providing this information! For anyone running a full installation of home assistant, as soon as you have a root shell to the host os, docker exec -it homeassistant bash (one dash less than above) should do it, as that's the default spelling on my instance.

epenet commented 1 year ago

I had to dig it up, but what I wanted to suggest is to edit instead this file: /usr/src/homeassistant/homeassistant/components/renault/manifest.json There you can just set "requirements": ["renault-api==0.1.13"]

I think it is better to match the patch PR :)

smgt commented 1 year ago

I had to dig it up, but what I wanted to suggest is to edit instead this file:

/usr/src/homeassistant/homeassistant/components/renault/manifest.json

There you can just set "requirements": ["renault-api==0.1.13"]

I think it is better to match the patch PR :)

So you mean that HA will automatically pull the new package? Or does it do it on restart?

epenet commented 1 year ago

It will automatically pick up the new package on restart.

I just think that using manifest.json instead of const.py mirrors better what will happen inside the next patch. If there is an issue (I don't expect any but you never know) it will be spotted earlier...

goberhammer commented 1 year ago

I had to dig it up, but what I wanted to suggest is to edit instead this file: /usr/src/homeassistant/homeassistant/components/renault/manifest.json There you can just set "requirements": ["renault-api==0.1.13"]

I think it is better to match the patch PR :)

Thanks for this, worked perfectly (homeassisntant on docker). Edited the file, restarted homeassistant from the UI and now my Twingo is back again. Thank again to everyone involved for solving this so quickliy :-)

JeedHome44 commented 1 year ago

Hello,

And those who have a normal HA can't do anything to explore the code and just modify the Manifest?

I don't want to wait several weeks.

I copied the Renault folder and put it in the custom folder but it is not taken into account in the integrations.

Just need to change 0.1.12 to 0.1.13

it would be great to have an update to change the version of Renault-api in the integration settings

it was simpler before when it was not integrated into the HA code

epenet commented 1 year ago

it would be great to have an update to change the version of Renault-api in the integration settings

As I have said, if you are extra motivated, you can create a PR to add an option flow so that next year the key can be set manually without waiting for a review process.

I will happily review a PR

JeedHome44 commented 1 year ago

Can we use File Editor to change manifest.json ?

JeedHome44 commented 1 year ago

it would be great to have an update to change the version of Renault-api in the integration settings

As I have said, if you are extra motivated, you can create a PR to add an option flow so that next year the key can be set manually without waiting for a review process.

I will happily review a PR

I would love to have this skill but I have no development knowledge like you

Tx3Roulio commented 1 year ago

Hi guys ! Thanks for your work on this addon :) How to edit the file manifest.json ? Because we can't access /usr/src/homeassistant with ssh ! Have a great day :)

hannut commented 1 year ago

@Tx3Roulio this is how I did it:

  1. install SSH & Web Terminal addon
  2. disable Protection mode from addon settings
  3. restart addon
  4. ssh to homeassistant
  5. docker exec -it homeassistant /bin/bash
  6. vi /usr/src/homeassistant/homeassistant/components/renault/manifest.json
    • change renault-api==0.1.12 to renault-api==0.1.13
    • save file
  7. restart homeassistant
JeedHome44 commented 1 year ago

@Tx3Roulio this is how I did it:

  1. install SSH & Web Terminal addon
  2. disable Protection mode from addon settings
  3. restart addon
  4. ssh to homeassistant
  5. docker exec -it homeassistant /bin/bash
  6. vi /usr/src/homeassistant/homeassistant/components/renault/manifest.json

    • change renault-api==0.1.12 to renault-api==0.1.13
    • save file
  7. restart homeassistant

i'm not using docker HA and i don't find option Protection mode in SSH addon. it's work only if docker ?

mathieucarbou commented 1 year ago

@Tx3Roulio this is how I did it:

  1. install SSH & Web Terminal addon
  2. disable Protection mode from addon settings
  3. restart addon
  4. ssh to homeassistant
  5. docker exec -it homeassistant /bin/bash
  6. vi /usr/src/homeassistant/homeassistant/components/renault/manifest.json

    • change renault-api==0.1.12 to renault-api==0.1.13
    • save file
  7. restart homeassistant

Not for HA OS....

JeedHome44 commented 1 year ago

no solution for HA OS ?

epenet commented 1 year ago

The solution works for HA OS, but you need to use the correct add-on Good => "SSH & Web Terminal" Bad => "Terminal & SSH"

JeedHome44 commented 1 year ago

The solution works for HA OS, but you need to use the correct add-on Good => "SSH & Web Terminal" Bad => "Terminal & SSH"

i will test immediatly !

JeedHome44 commented 1 year ago

sudo nano don't work ? i never use bash. when i tap delete and 3 to change 2, nothing append

{
  "domain": "renault",
  "name": "Renault",
  "integration_type": "hub",
  "config_flow": true,
  "documentation": "https://www.home-assistant.io/integrations/renault",
  "requirements": ["renault-api==0.1.12"],
  "codeowners": ["@epenet"],
  "iot_class": "cloud_polling",
  "loggers": ["renault_api"]
}
~
~
~
~
~
~
~
~
~
~
~
~
- /usr/src/homeassistant/homeassistant/components/renault/manifest.json 7/11 63%

someone can say how modify file and save ? thank you

Edit: i find to change 12 > 13 but how save file please ?

epenet commented 1 year ago

nano isn't available currently. Look on the web for tutorials on using vi

epenet commented 1 year ago

I have HA OS and even with "SSH & Web Terminal" this is not working and I do not have access at all to the HA system.

You didn't run step 5

5. docker exec -it homeassistant /bin/bash

mathieucarbou commented 1 year ago

thansk a lot! I was using the wrong ssh addon....

... and forgot the docker command.

JeedHome44 commented 1 year ago

thank you ! I found my zoe ! it's work !

jladefoged commented 1 year ago

Looks like the issue is solved in the dev version of Home Assistant and the fix will be in the 2023.4.6 milestone release.

PaulMuadDib commented 1 year ago

it_IT back online ! thanks @epenet for pointing the good/bad ssh add-on ! la zoé ronronne à nouveau dans H.A. :o))

Tx3Roulio commented 1 year ago

@Tx3Roulio this is how I did it:

  1. install SSH & Web Terminal addon
  2. disable Protection mode from addon settings
  3. restart addon
  4. ssh to homeassistant
  5. docker exec -it homeassistant /bin/bash
  6. vi /usr/src/homeassistant/homeassistant/components/renault/manifest.json

    • change renault-api==0.1.12 to renault-api==0.1.13
    • save file
  7. restart homeassistant

Thanks a lot, it works like a charm :)

Paps42 commented 1 year ago

fantastic, it worked for me on HA OS, with this procedure: " install SSH & Web Terminal addon disable Protection mode from addon settings restart addon ssh to homeassistant docker exec -it homeassistant /bin/bash vi /usr/src/homeassistant/homeassistant/components/renault/manifest.json change renault-api==0.1.12 to renault-api==0.1.13 save file restart homeassistant " you are the best, thank you so much.

My Zoé is now to return, i'm so happy

MilorES commented 1 year ago

imagen Are you sure the folder is correct?

epenet commented 1 year ago

Are you sure the folder is correct?

You didn't run step 5

5. docker exec -it homeassistant /bin/bash