ReneNulschDE / mbapi2020

Custom Component to integrate MercedesME devices into Home-Assistant
Other
149 stars 29 forks source link

European Market - Pin Email Request Not Working #184

Closed hcraveiro closed 7 months ago

hcraveiro commented 7 months ago

Hi,

For the past week I stopped receiving PIN whenever I need to configure mbapi2020. I tried logging out of Mercedes Me app and logging back in to see if I receive the PIN, and yes, I receive the PIN on the email. I don't have any special characters on the email (I use email). I have devices on my network where I can run commands (namely the homeassistant OS).

This is my info: Version core-2024.1.0 Installation Type Home Assistant OS Development false Supervisor true Docker true User root Virtual Environment false Python Version 3.11.6 Operating System Family Linux Operating System Version 6.1.63-haos-raspi CPU Architecture aarch64 Timezone Europe/Lisbon Configuration Directory /config

Home Assistant Community Store GitHub API ok GitHub Content ok GitHub Web ok GitHub API Calls Remaining 5000 Installed Version 1.34.0 Stage running Available Repositories 1379 Downloaded Repositories 4

Home Assistant Cloud Logged In false Reach Certificate Server ok Reach Authentication Server ok Reach Home Assistant Cloud ok

Home Assistant Supervisor Host Operating System Home Assistant OS 11.4 Update Channel stable Supervisor Version supervisor-2023.12.1 Agent Version 1.6.0 Docker Version 24.0.7 Disk Total 14.0 GB Disk Used 8.3 GB Healthy true Supported true Board rpi3-64 Supervisor API ok Version API ok Installed Add-ons Samba share (12.2.0), Terminal & SSH (9.8.1), Whisper (1.0.2), Piper (1.4.0), Duck DNS (1.15.0)

Dashboards Dashboards 2 Resources 0 Views 11 Mode storage

MercedesME 2020 error unknown

manifest.json: { "domain": "mbapi2020", "name": "MercedesME 2020", "codeowners": [ "@ReneNulschDE" ], "config_flow": true, "dependencies": [], "documentation": "https://github.com/ReneNulschDE/mbapi2020", "integration_type": "hub", "iot_class": "cloud_push", "issue_tracker": "https://github.com/ReneNulschDE/mbapi2020/issues", "loggers": [ "custom_components.mbapi2020" ], "requirements": [ "protobuf>=3.19.1" ], "version": "0.9.9" }

ReneNulschDE commented 7 months ago

Thanks for creating an extra issue.

Lets dive a little bit deeper...

I checked the image with my account and it works as expected...

  1. connect to your SSH-Addon Shell connect directly to the HomeAssistant instance with "docker exec -it homeassistant bash"
  2. Copy the curl command from https://pastes.io/0uugksd1bd
  3. Replace XXXXXX with your mb-account-email
  4. run
  5. check the result
  6. leave the docker instance with "exit" Result should be:
    
    HTTP/1.1 200 OK
    Content-Security-Policy: default-src https:
    Content-Type: application/json
    Referrer-Policy: strict-origin-when-cross-origin
    Strict-Transport-Security: max-age=63072000
    X-Authmode: CIAMNG
    X-Content-Type-Options: nosniff
    X-Frame-Options: SAMEORIGIN
    X-Permitted-Cross-Domain-Policies: none
    Date: Sun, 28 Jan 2024 19:12:51 GMT
    Content-Length: 43
    Connection: close

{"isEmail":true,"username":"XXXXXXX"}


and you should get a email...

In case you got an eror (http400) and did not got an email. Run the same steps without connecting to the homeassistant-core instance. (so just in the SSH-Addon-shell)

If also this fails, please check if you can use a device in your local network but not connected to HAOS. In case this is a windows machine let me know - I have an powershell command with the same content available.
hcraveiro commented 7 months ago

Question about 1) I connected through SSH to homeassistant instance. I run 'docker' but it says there no such command. Is this normal?

ReneNulschDE commented 7 months ago

I have no experience with the ssh/telnet addin that you use. I assume that you directly connect to the homeassistant core instance. You can check with the hostname command. When the result is "homeassistant" that you are already on the homeassistant core instance.

I use the standard "Advanced SSH & Web Terminal" (17.xxx) and with that you connect to a special environment.

hcraveiro commented 7 months ago

Doing hostname gives me 'core-ssh', so it should be homeassistant core. On this host I have access to config folder, from where I got the info from manifest.json

ReneNulschDE commented 7 months ago

Looks like this is a special instance and the config folder is just linked in. And this addon has no docker tools installed.

But you said that the response was http200 with empty email address, right?

Do you have another pc where you can this command? I have a windows version based on powershell available too if needed.

hcraveiro commented 7 months ago

I just tried running the command I I got nothing as response, just empty.

ReneNulschDE commented 7 months ago

No output at all? or was it like

HTTP/1.1 400 ...
...
...
...
hcraveiro commented 7 months ago

No output at all, not even HTTP code.

ReneNulschDE commented 7 months ago

Please copy it again from the source and change the email address.

hcraveiro commented 7 months ago

I changed the email address. I tried now to do this without connecting to the homeassistant core through SSH and got this: HTTP/1.1 200 OK Content-Security-Policy: default-src https: Content-Type: application/json Referrer-Policy: strict-origin-when-cross-origin Strict-Transport-Security: max-age=63072000 X-Authmode: CIAMNG X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN X-Permitted-Cross-Domain-Policies: none Date: Mon, 29 Jan 2024 17:01:49 GMT Content-Length: 31 Connection: close

{"isEmail":true,"username":""}

hcraveiro commented 7 months ago

I tried with the phone number and got: HTTP/1.1 200 OK Content-Security-Policy: default-src https: Content-Type: application/json Referrer-Policy: strict-origin-when-cross-origin Strict-Transport-Security: max-age=63072000 X-Authmode: CIAMNG X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN X-Permitted-Cross-Domain-Policies: none Date: Mon, 29 Jan 2024 17:03:18 GMT Content-Length: 32 Connection: close

{"isEmail":false,"username":""}

ReneNulschDE commented 7 months ago

Give me a few minutes, I want to check something and have to reproduce your environment. What is your country of living? Portugal?

ReneNulschDE commented 7 months ago

Do you have a maschine not connected to HA, where you could test the statement? I also have one for a windows maschine availabe?

If this does not work... OK - lets try what others have done:

  1. please go in the mb app - and change the security pin (you can use the same again)
  2. test mail
  3. please login on the mercedes-benz website and check if you get any new data protection topics.
  4. test mail
  5. if not done, delete the integration via the frontend/hacs depending on your install method
  6. delete the file ".mercedesme-token-cache" in the config folder with "rm /config/.mercedesme-token-cache"
  7. restart HA
  8. install the mbapi component again
  9. try to login
hcraveiro commented 7 months ago

I tried that sequence and when I try to install the component, I put my email (or phone) and press submit key I get "Unknown error occurred". I received the emails from the app or website before...

ReneNulschDE commented 7 months ago

What is in the log output?

hcraveiro commented 7 months ago

Logger: custom_components.mbapi2020.oauth Source: custom_components/mbapi2020/oauth.py:224 Integration: MercedesME 2020 (documentation, issues) First occurred: 8:20:57 PM (2 occurrences) Last logged: 8:39:42 PM ClientError requesting data from https://bff.emea-prod.mobilesdk.mercedes-benz.com/v1/login: Cannot connect to host bff.emea-prod.mobilesdk.mercedes-benz.com:443 ssl:True [Name has no usable address]

ReneNulschDE commented 7 months ago

This is sounds like an DNS or networking error. Is your DNS Server reachable? check nslookup bff.emea-prod.mobilesdk.mercedes-benz.com expected result: Address: 40.68.60.81

hcraveiro commented 7 months ago

If I do the nslookup from my laptop it works fine, if I go to terminal in home assistant and do it there it says that it can't find the host. DO I need to do anything on Home Assistant? It's Home Assistant OS...

ReneNulschDE commented 7 months ago

I'm not an HAOS specialist but I would restart the complete HAOS. Looks like the system has some problems with DNS.

ReneNulschDE commented 7 months ago

Could you please reset the security pin in your MB mobile app and check again? This was the solution for multiple users.

svinginum commented 7 months ago

I can confirm that changing pin in MercedesME App helped.

modem-man-gmx commented 7 months ago

Hi Rene, could you please repeat the Curl command? The link you wrote on top a week before is stale.

I think I have the same issue.

bff.emea-prod.mobilesdk.mercedes-benz.com resolves to 40.68.60.81 but a simple call of curl bff.emea-prod.mobilesdk.mercedes-benz.com is always returning curl: (28) Failed to connect to bff.emea-prod.mobilesdk.mercedes-benz.com port 80 after 129591 ms: Couldn't connect to server after 2 minutes. This happens on the Home Assistant hardware as well as on my desktop PC.

More interestingly, "dig id.mercedes-benz.com", taken from mbapi2020/const.py resolves to

id.mercedes-benz.com.   45      IN      A       18.158.241.79
id.mercedes-benz.com.   45      IN      A       3.66.15.222
id.mercedes-benz.com.   45      IN      A       52.59.65.184

but always returning this to curl:

$ curl "id.mercedes-benz.com"
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
</body>
</html>

Can't say, if this is helping us somehow.

EDIT: I repeated the "+ Add Integration" > MB2020 > email, and captured a tcpdump. There is a 29 packets communication with 40.86.60.81 via TLS1.3, but I forgot how the man in the middle decryption is working. So the 40.86.60.81 is at least talking with me, no DNS or firewall issue. Need to repeat this with your "curl" command or to fiddle out the MITM method ... tomorrow.

modem-man-gmx commented 7 months ago

@svinginum, you wrote:

I can confirm that changing pin in MercedesME App helped.

Could you please tell me, if I did the same compared to you?

  1. Opening mobile phone Mercedesme App,
  2. opening the Hamburger menu
  3. at top verify / note the own mail address
  4. at bottom -> Settings
  5. Security | Mercedes me PIN >
  6. entered old PIN, entered a new PIN
  7. Save
  8. switch over to Home Assistant
  9. Opening Settings | Integrations
  10. bottom righthand "+ Add Integration"
  11. select brand: "mercedes", pick "MercedesME 2020 >"
  12. for "MB username - email address (or mobile ...) entering the address verified with topic 3
  13. Region (*) Europe
  14. "Send"
  15. got [or not got] the email from MB with 6-digits

same at your side? I had no luck with topic 15 :-(

ReneNulschDE commented 7 months ago

Here the link to the curl command https://homeass.ist/?da510b0883c3e91b#947sDS6j39bd1N9SYLvGhx9r6EhSANHDsFNmVb4WTF7e

Dont forget to replace XXXXXX with your mb-login-email (in case you registered first with mobile then please use the mobile number)

ReneNulschDE commented 7 months ago

If you would like to play with MITM. Take a look in the const.py. There is already an option to enable/set a proxy and to disable the SSL-cert-checks.

And you said that you get an HTTP400. This means networking is ok, Cert checks are ok...

Just connecting to port80 makes no sense. Its on 443. Please do not play with this endpoint. I'm happy that the MB team has not stopped what I have developed with their legal department.

But would be good to see the logs from your MITM-proxy.

talpazzo commented 7 months ago

I'm from Switzerland, I had the same problem. No email was received at the configuration. Changed the pin in the app (my pin -> temp one > my old pin) at this point I restarted the configuration and an email with the pin was received.

modem-man-gmx commented 7 months ago

Hi @ReneNulschDE ,

Here the link to the curl command

Tried the Curl, got 200 OK. Not sure if the last two lines are expected? result here: https://homeass.ist/?a1fe6972d3dce597#JAFamszVfPFupUfmb7bZN4P127hBmVLHQDDcRfoM47eD

Changing the PIN once more did not work for me. Even with waiting several 10-minutes between PIN and Setup. And even with visiting M-me in the browser in the meantime :-(

I think it's not worth setting up a MITM proxy, because the Curl should also had triggered a 6-digit-code commision, right?

One thing is different, compared to most other drivers: The owner (me) only registered the email at mb.com, but no Cell Phone (don't own such). Or to be more precise: one of my childs registered the car and granting access to the mobile phone to me. And later we added 2 Co-drivers, both with mail and mobile number. One of the Co-Drivers has been already an M-me user, registered with his primary car. All of this was made before 1st Home Assistant usage - just to note. Only pictures for the MBUX greeting screen have been uploaded or changed since.

svinginum commented 7 months ago

@svinginum, you wrote:

I can confirm that changing pin in MercedesME App helped.

Could you please tell me, if I did the same compared to you?

  1. Opening mobile phone Mercedesme App,
  2. opening the Hamburger menu
  3. at top verify / note the own mail address
  4. at bottom -> Settings
  5. Security | Mercedes me PIN >
  6. entered old PIN, entered a new PIN
  7. Save
  8. switch over to Home Assistant
  9. Opening Settings | Integrations
  10. bottom righthand "+ Add Integration"
  11. select brand: "mercedes", pick "MercedesME 2020 >"
  12. for "MB username - email address (or mobile ...) entering the address verified with topic 3
  13. Region (*) Europe
  14. "Send"
  15. got [or not got] the email from MB with 6-digits

same at your side? I had no luck with topic 15 :-(

II did delete the authentication token as step 1 in the Homeassistant

ReneNulschDE commented 7 months ago

Thanks, @modem-man-gmx ,

Did you changed the result in any way especially line67? If not than this request was not successful. The normal behavior is that the attribute email is filled with same value like in the request.

Could you initiate the curl command outside of HA on a different Maschine?

Are you using a secondary driver account or the primary one now?

ArBohe commented 7 months ago

Hi Everyone ! Same for me in France - No pin received. I ran the curl command and got the same as @modem-man-gmx (no email re-suplied at line 67)

* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Content-Security-Policy: default-src https:
Content-Security-Policy: default-src https:
< Content-Type: application/json
Content-Type: application/json
< Referrer-Policy: strict-origin-when-cross-origin
Referrer-Policy: strict-origin-when-cross-origin
< Strict-Transport-Security: max-age=63072000
Strict-Transport-Security: max-age=63072000
< X-Authmode: CIAMNG
X-Authmode: CIAMNG
< X-Content-Type-Options: nosniff
X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
X-Frame-Options: SAMEORIGIN
< X-Permitted-Cross-Domain-Policies: none
X-Permitted-Cross-Domain-Policies: none
< Date: Mon, 05 Feb 2024 06:14:56 GMT
Date: Mon, 05 Feb 2024 06:14:56 GMT
< Content-Length: 31
Content-Length: 31
< Connection: close
Connection: close

<
{"isEmail":true,"username":""}
* Closing connection 0
* TLSv1.3 (IN), TLS alert, close notify (256):
* TLSv1.3 (OUT), TLS alert, close notify (256):

for info, I had the integration working well and it stopped working Saturday. looked for the cause, didn't find, uninstalled, re-installed and I'm stuck at not getting a pin through email.

ArBohe commented 7 months ago

Addition to my post above. So yes, the curl works from another network (pin received). (Cannot test yet on another machine, same network as I'm remote). So the issue seem to be networked related, yet new from 2 days without any changes whatsoever of the network nor HA setup.

Cheers!

ReneNulschDE commented 7 months ago

Thx, could you please share the details of youre HA system and highlevel the working system?

modem-man-gmx commented 7 months ago

@ReneNulschDE asked:

Did you changed the result in any way especially line67? If not than this request was not successful. The normal behavior is that the attribute email is filled with same value like in the request.

No, I did not change a single bit.

Could you initiate the curl command outside of HA on a different Maschine?

Yes, here we go, it's Win10 with MSYS2 Bash (Arch Linux like): https://homeass.ist/?ba6bc16dfecc4fa4#4vd7LgKazf46zUWG5MXWvqA87JoJiaFf2A15FiTHw77q Nothing changed, just repeated Your curl line with replaced XXXXXXX with primary owner's gmail address.

Are you using a secondary driver account or the primary one now?

The account is the primary driver. The email is same as used on the Mobile, and is the only one who sees also the car depicted under "my Cars" at the MB-me web page.

modem-man-gmx commented 7 months ago

Thx, could you please share the details of youre HA system and highlevel the working system?

HA Core 2024.1.4 Frontend 20240104.0 HACS Integrations Version: | 1.34.0 HACS Frontend Version: | 20220906112053 $ uname -a Linux hassi 6.1.0-rpi7-rpi-v8 #1 SMP PREEMPT Debian 1:6.1.63-1+rpt1 (2023-11-24) aarch64 GNU/Linux $ docker -v Docker version 24.0.7, build afdd53b container: lscr.io/linuxserver/homeassistant:latest LABEL build_version=Linuxserver.io version:- 5afef06a-ls34 Build-date:- 2024-01-13T13:29:33+00:00

Home Assistant is not running on the Docker NAT bridge, but on an own vlan interface.

(sorry for many edits, I always miss the full CR/LF after a quote)

ReneNulschDE commented 7 months ago

@modem-man-gmx : Could you send me an email? mb.dev.i184@nulsch.de - I have a non-public question.

ArBohe commented 7 months ago

Some info:

Type Value
Version core-2024.1.6
Installation Type Home Assistant OS 11.5
Developpement false
Supervisor true
Docker true
User root
Environnement virtuel false
Python version 3.11.6
Famille du système d'exploitation Linux
Version du système d'exploitation 6.1.63-haos-raspi
Architecture du processeur aarch64
Fuseau horaire Europe/Paris
Répertoire de configuration /config
Type Value
GitHub API ok
GitHub Content ok
GitHub Web ok
GitHub API Calls Remaining 5000
Installed Version 1.34.0
Stage running
Available Repositories 1395
Downloaded Repositories 26
HACS Data ok
Type Value
Connecté false
Accéder au serveur de certificats ok
Accéder au serveur d'authentification ok
Accéder à Home Assistant Cloud ok
Type Value
Système d'exploitation hôte Home Assistant OS 11.5
Canal de mise à jour stable
Version du Supervisor supervisor-2023.12.1
Version de l'agent 1.6.0
Version de Docker 24.0.7
Taille total du disque 113.6 GB
Taille du disque utilisé 13.7 GB
Sain true
Prise en charge true
Tableau de bord rpi4-64
API du Supervisor ok
Version de l'API ok
Modules complémentaires installés File editor (5.7.0), Advanced SSH & Web Terminal (17.1.0), Mosquitto broker (6.4.0), MyElectricalData (0.9.3), Tailscale (0.15.0), go2rtc (1.8.5), InfluxDB (5.0.0), Ring-MQTT with Video Streaming (5.6.3), Zigbee2MQTT (1.35.2-1), Matter Server (5.0.4), ESPHome (2023.12.9), SwitchBot-Mqtt (1.0.8), Duck DNS (1.15.0), NGINX Home Assistant SSL proxy (3.7.0), Home Assistant Google Drive Backup (0.112.1)
Type Value
Dashboard 1
Ressources 6
Vues 12
Mode storage
ArBohe commented 7 months ago

Hot off the press!!! it now does not work either on a different instance, different network (the CURL command sends back {"isEmail":true,"username":""} as the issue describer above. So not going in the right direction, but this is a sign of generalization, unlink to the HA instance (this one for sure was not changed in ANY way since yesterday when it was functioning.

Thanks for your help btw !

ADDITIONAL UPDATE

Tested on a mac, no HA installed - just running CURL - on yet another different location (different network): empty also.

ReneNulschDE commented 7 months ago

Thanks @ArBohe ,

this supports my general assumption that it is not a problem with HA nor the integration (as I and multiple other get the pins) and even users of other home automation systems have the same problem.

May I ask you to support a fresh start:

  1. Delete the integration from HA.
  2. Delete the token file /config/.mercedesme-token-cache (This only valid for MBAPI2020 < v0.10.0, and yes this version does not exists yet (at least on HACS) but to make sure that future readers dont do a wrong thing. Starting v0.10 multi-user support is integrated and token handling will change)
  3. Restart HA
  4. Open the MB-Mobile App and reset the security pin
  5. Add the integration again with the same account and login method (email/phone) that you use in the app
  6. report back if you received an email.

Update:

  1. In case you have not received an email. Re-run the curl and export the complete output. You can share it via homeass.ist or send it to mb.deb.i184@nulsch.de (without your account info of course)

thx

ArBohe commented 7 months ago

Just followed the process: no joy -- I sent the CURL output to your email

ReneNulschDE commented 7 months ago

Short update: I can reproduce the issue now. Please be patient, I'm working on it.

@ArBohe, @modem-man-gmx : Thanks for your time yesterday and all tests.

ReneNulschDE commented 7 months ago

I have created a beta release v0.9.11b3. This should solve the Pin issue. Feedback is welcome.

You can test it with HACS:

  1. Click on Update Information

image

  1. Click on Redownload and activate the switch "Show beta versions"

image

  1. Select the version v0.9.11.b3, click "Download"
  2. Don't forget the restart HA before starting the Pin flow.
modem-man-gmx commented 7 months ago

Great!! version v0.9.11.b3 does it here now!

image

MERCI!

hcraveiro commented 7 months ago

Great, it works now! Thank you!

krisblackhall commented 7 months ago

I can confirm this worked for me too. Updated to the latest beta and now I am once again getting the MB PIN email coming through within 10 seconds of entering email address when setting up the integration. Many thanks.