403 Client Error: Forbidden for url: https://www.wemportal.com/app/Account/Login

[dm82m]

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: custom_components.wemportal Quelle: custom_components/wemportal/coordinator.py:67 Integration: Weishaupt WEM Portal (Dokumentation, Probleme) Erstmals aufgetreten: 07:13:29 (5 Vorkommnisse) Zuletzt protokolliert: 07:16:01

Error fetching data from wemportal Traceback (most recent call last): File "/config/custom_components/wemportal/wemportalapi.py", line 195, in api_login response.raise_for_status() File "/usr/local/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.wemportal.com/app/Account/Login

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/config/custom_components/wemportal/wemportalapi.py", line 88, in fetch_data self.api_login() File "/config/custom_components/wemportal/wemportalapi.py", line 208, in api_login raise ForbiddenError( custom_components.wemportal.exceptions.ForbiddenError: WemPortal forbidden error: Server returned internal status code: and message:

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/config/custom_components/wemportal/coordinator.py", line 67, in _async_update_data x = await self.hass.async_add_executor_job( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/wemportal/wemportalapi.py", line 142, in fetch_data raise WemPortalError from exc custom_components.wemportal.exceptions.WemPortalError

[dm82m]

It can’t be a block as I can access the wemportal aswell as the app without any problems.

[dm82m]

@erikkastelec can you investigate here?

[developer-stephan]

The same error since today morning...

[dm82m]

Also tried to set mode to web to overcome using app endpoint but it’s not helping. I guess they are blocking the headers that are used here. But we need to wait for Erik for a detailed analysis.

[mirk-daniel]

Same problem since this mornig.

[developer-stephan]

Maybe with a header like "Mozilla/5.0 Gecko/20100101 Firefox" could help :)

[Thin-White-Duke]

I get the same error here since yesterday ... WEM portal access still working fine.

[chriscom2020]

Yes, same error since 2.00am (GMT+2). WEM Portal still accessible...

[dm82m]

Let’s try to keep this issue clean: more „me too“ reply’s aren’t helping. It’s a global issue affecting all users. Just wait for Erik or start investigating on your own and share new insights.

[developer-stephan]

I think the problem is there:

https://github.com/erikkastelec/hass-WEM-Portal/blob/5c213911e35a459fb1d2dd07c14ac97ccaa30e2e/custom_components/wemportal/wemportalapi.py#L73

Maybe I will check it tonight.

[erikkastelec]

I am experiencing the same issue. I will try to fix this tomorrow afternoon.

[dm82m]

I think the problem is there:

https://github.com/erikkastelec/hass-WEM-Portal/blob/5c213911e35a459fb1d2dd07c14ac97ccaa30e2e/custom_components/wemportal/wemportalapi.py#L73

Maybe I will check it tonight.

Tested your alternative user agent from above. Same issue. I guess we need to debug the app to see what requests they are sending out. But honestly there was no app update. So user agent should be the same. But anyhow. Maybe they are explicitly blocking the requests coming from this I integration.

[chriscom2020]

https://www.wemportal.de/Web/ stopped working as well...

[dm82m]

https://www.wemportal.de/Web/ stopped working as well...

works for me, URL is accessible and login is possible

[mirk-daniel]

https://www.wemportal.de/Web/ stopped working as well...

I had this problem too (HTTP Error 403), but after clicking refresh several times the website will work again.

Sometimes the app shows old values today or the login is not possible.

Today it’s a little strange.

[dm82m]

I tried to ssl debug the app connection, but I cant as my iphone is a business iphone and does not allow me to import certificates. Other possibility would be to use an emulator, but thats to much effort at the moment. Just waiting for master erik :D

[developer-stephan]

I just checked. It looks like they added "__VIEWSTATEGENERATOR" or something new in:

<div class="aspNetHidden">

Thats why the browser and the app (WebView) can get the new form field(s) and submit that to the login. Nothing to do with User Agent or SSL.

Just an idea...

[Karo-X]

I myself have problems establishing a connection with the iOS app at irregular intervals. Here, however, the error message was ambiguous - “Login failed! Username / password could not be found. ”. On the website I get the 403 message. I am currently unsure whether this is perhaps also a DDoS prevention on the part of Weishaupt. Because if you try it with both the app / and the website at regular intervals, it works. I wrote to Weishaupt yesterday to possibly make a whitelist request (I have a fixed IP address).

[thuesen]

If DDoS prevention is the issue here or not, I don’t know. But it does seem they have something in that regards (don’t know if it’s new or not).

I tried with the iOS app to launch it, then force quit, launch again, force quit, and after 5-6 attempts in a row, I got error prompt that my username/password could not be found. Then I switched IP and tried again, and login worked. Switched back to previous IP, and login failed.

Either way, it’s probably a good idea to ensure this (awesome!) integration doesn’t make more requests to Weishaupt than absolutely necessary, so we don’t give them a reason to actively work against this integration.

[Karo-X]

It is strange. Especially when I reload the website several times in a row, the page is loaded incompletely (some icons are missing) or the layout is completely shifted and then ends in the 403 continue training output. I will also contact Weishaupt by phone.

[mirk-daniel]

Hey, I know this question is a little bit off-topic.....

Does anybody tried the JSON-interface? In the network-settings (Weishaupt WTC) it is possible to activate a "JSON interface" but I don't find any more information about it. If we get the information through the JSON-Interface we don't need the WEM-Protal ;)

[dm82m]

Hey, I know this question is a little bit off-topic.....

Does anybody tried the JSON-interface? In the network-settings (Weishaupt WTC) it is possible to activate a "JSON interface" but I don't find any more information about it. If we get the information through the JSON-Interface we don't need the WEM-Protal ;)

Create a new issue here and describe a little more what you found, how and were it can be activated, post some screenshots. Local interface would be a game changer for this *** Weishaupt software…

[developer-stephan]

It sounds its just for the diagnosis. That's all. I do not think we can change something there like room temperature. Its off-topic.

[misterbond007]

This is called "Modbus TCP" and can be activated also in the WEM Portal. See enclosed screenshots. Unfortunately, there is no such integration for Home Assistant. I indeed think, the data for this Modbus TCP differs from the type of the Weishaupt heating pump.

https://community.home-assistant.io/t/weishaupt-heatpump-integration-via-modbus/436823

[dm82m]

Maybe an idea to try to use the modbus tcp to get data. But not sure if that interface provides the data we need

[misterbond007]

After some searching, I found two HACS integrations for Modbus TCP integration. They are currently in beta: https://github.com/MadOne/weishaupt_modbus https://github.com/OStrama/weishaupt_modbus

[dm82m]

Worth a try we

[developer-stephan]

Unfortunately I cannot find Modbus TCP in my WEM Portal account. I have a gas burner. Not a heat pump :-(

[juergen88400]

My gas burner "WTC" offers only WEMPortal. There is a "Gateway WEM to modbus" (729,- EUR !!!) that switches communication to modbus. But WEM Portal is then no longer available - which as a consequence no longer allows to use the portal on your PC or the app on your smartphone. Not sure whether this is what I want. Poor integration on weishaupt side. I'll consider this once my WTC needs to be replaced.

[erikkastelec]

I know this issue affects a lot of you, so I will try my best to fix it this weekend. Due to some complications at work, I wasn't able to get to it yet.

[misterbond007]

I know this issue affects a lot of you, so I will try my best to fix it this weekend. Due to some complications at work, I wasn't able to get to it yet.

Thank you very much Erik!

[ghenzler]

I did some debugging on my side... the reason why the mode web does not help is that it still loads some data from the API (in particular the ID of the device), this is here: https://github.com/erikkastelec/hass-WEM-Portal/blob/5c213911e35a459fb1d2dd07c14ac97ccaa30e2e/custom_components/wemportal/wemportalapi.py#L87 (I disabled these lines with mode web and then the web part comes in, but with a wrong ID)

So the root cause is the App API. I tested with this curl request that replicates exactly what python does (but for me it's always a bit easier to run curl to do some tweaks quickly):

curl -v https://www.wemportal.com/app/Account/Login \
  -X POST \
  -H "User-Agent: WeishauptWEMApp" \
  -H "X-Api-Version: 2.0.0.0" \
  -H "Accept: */*" \
  -d '{"Name": "MY_USER ","PasswordUTF8":"MY_PASSWORD","AppID":"com.weishaupt.wemapp","AppVersion": "2.0.2","ClientOS": "Android"}'

For that I currently get mostly 403 Forbidden (but interesting enough sometimes also 400 Bad Request). I think User-Agent should really be ok here as it has worked for a long time. I tried with a newer API version or app version and also a "browser-like" user agent, but that all didn't work.

From the "real app" I also had problems logging on that last hour or so, but while I'm writing this I just got a valid session in the app itself again (curl gives me the same problem though)

@erikkastelec How did you do your reverse-engineering on how the app makes requests to the server? I think this App-Login request needs to be tweaked somehow. Or maybe Weishaupt just has server problems and the problem will go away itself.

[mbuchloh]

I'm not shure if it helps, or is already known. Access to WEM Portal (web) is OK. Log in HA is this: 2024-10-06 00:00:07.140 DEBUG (SyncWorker_57) [custom_components.wemportal] Fetching api device data 2024-10-06 00:00:07.140 DEBUG (SyncWorker_57) [custom_components.wemportal] Sending GET request to https://www.wemportal.com/app/device/Read with headers: {'User-Agent': 'WeishauptWEMApp', 'X-Api-Version': '2.0.0.0', 'Accept': '/'} 2024-10-06 00:00:07.175 WARNING (MainThread) [custom_components.wemportal] Creating new wemportal api instance Traceback (most recent call last): File "/config/custom_components/wemportal/wemportalapi.py", line 254, in make_api_call response.raise_for_status() File "/usr/local/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.wemportal.com/app/device/Read

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/config/custom_components/wemportal/wemportalapi.py", line 90, in fetch_data self.get_devices() File "/config/custom_components/wemportal/wemportalapi.py", line 279, in get_devices data = self.make_api_call("https://www.wemportal.com/app/device/Read").json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/wemportal/wemportalapi.py", line 269, in make_api_call raise WemPortalError( custom_components.wemportal.exceptions.WemPortalError: An error occurred while gathering data.This issue should resolve by itself. If this problem persists,open an issue at https://github.com/erikkastelec/hass-WEM-Portal/issues Server returned status code: and message:

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/config/custom_components/wemportal/coordinator.py", line 47, in _async_update_data x = await self.hass.async_add_executor_job(self.api.fetch_data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/wemportal/wemportalapi.py", line 142, in fetch_data raise WemPortalError from exc custom_components.wemportal.exceptions.WemPortalError 2024-10-06 00:00:07.183 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 0.238 seconds (success: False)

[Karo-X]

I'm not shure if it helps, or is already known. Access to WEM Portal (web) is OK.

How can you say "Access to WEM Portal (web) is OK."? Or how do you determine this? I still get access denied.

[ghenzler]

Meanwhile I used a request monitoring app on iOS and using the following curl request I get sometimes (yes, only sometimes!) 200 OK now for:

curl -v https://www.wemportal.com/app/Account/Login \
  -X POST \
  -H "User-Agent: WeishauptWEMApp" \
  -H "X-Api-Version: 2.0.0.0" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Accept-Language: de-DE" \
  -d '{"Name": "MY_USER","PasswordUTF8":"MY_PASSWORD","AppID":"com.weishaupt.wemapp","AppVersion": "2.5","ClientOS": "Android"}'

This is reflected also in PR https://github.com/ghenzler/hass-WEM-Portal/commit/8236fca7562faf27c8e8b9274b6c24f1fcdf14a0 - but I have not been lucky with HA to get through with a 200 - so far only 403. I think overall it's the Azure infrastructure with rate limits (and potentially some rules I don't fully understand).

@erikkastelec I hope this helps for analysis at least - maybe you have an idea on how to fix it?

[mbuchloh]

I'm not shure if it helps, or is already known. Access to WEM Portal (web) is OK.

How can you say "Access to WEM Portal (web) is OK."? Or how do you determine this? I still get access denied.

I just log in (https://www.wemportal.com/Web/Default.aspx) and it's OK:

[ghenzler]

@mbuchloh you logged in to the wem portal in the browser, but the part that fails is the login via API (same as the mobile app does, and as described in my first comment above, even when on method web, the integration needs to retrieve the device ID via API first, and that requires an API login).

[mbuchloh]

Thats what I meant. The point I want to transport: The issue is: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.wemportal.com/app/Account/**Login I get: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.wemportal.com/app/device/Read**

Is my Read-request after login? or before?

[tech2mar]

Is it possible that there's a difference between the firmwares? Even though I'd like to use Modbus, I can't find it in any of the settings (11 or 21)—no Modbus for me. The Modbus part is actually well documented; it's just a bunch of YAML, and it doesn't seem that bad. But still, I paid for a premium heat pump, and they keep giving me very poor firmware and software solutions.

[erikkastelec]

@ghenzler I decompiled Android app to see what endpoints are being used. Here are quick instructions I followed link. I will look into headers in your pull request and test it.

Currently I have no problem logging, but get 403 when wanting to read data. Maybe endpoints changed or there were some additional headers added.

[developer-stephan]

@erikkastelec Currently I have no problem logging, but get 403 when wanting to read data. Maybe endpoints changed or there were some additional headers added.

Yes I agree with you. I tested it in Node-RED. I think the endpoints changed.

[erikkastelec]

@developer-stephan I have found out that /app/Device/Read works while /app/device/Read produces 403.

[mbuchloh]

Great, thanks a lot. With "Device" everything is fine again. It took some time until entities were available, but after some minutes everything worked.

[misterbond007]

@developer-stephan I have found out that /app/Device/Read works while /app/device/Read produces 403.

Unbelievable! One letter and everything is working again! Thank you very much Erik (hvala lepa ;-) ) and everybody!

[ManuelBu-dev]

Hi! Thanks for working in the issue. Quite new on HA. Where do I have to change the letter? Many thanks!!

[developer-stephan]

Hi! Thanks for working in the issue. Quite new on HA. Where do I have to change the letter? Many thanks!!

In the code wemportalapi.py replace "device" with "Device". See the screenshot. Then reload the HA :)

[misterbond007]

Hi! Thanks for working in the issue. Quite new on HA. Where do I have to change the letter? Many thanks!!

In the code wemportalapi.py replace "device" with "Device". See the screenshot. Then reload the HA :)

It‘s in the line 279 in that file. :-)

[edlestau]

Thanks a lot!

[ManuelBu-dev]

Hi! Thanks for working in the issue. Quite new on HA. Where do I have to change the letter? Many thanks!!

In the code wemportalapi.py replace "device" with "Device". See the screenshot. Then reload the HA :)

It‘s in the line 279 in that file. :-)

Hi, sorry I don´t know where the issue is - probably me. But same issue after restarting many times and also deleting and re installing integration:

Can you advise again please?

[misterbond007]

Hi! Thanks for working in the issue. Quite new on HA. Where do I have to change the letter? Many thanks!!

In the code wemportalapi.py replace "device" with "Device". See the screenshot. Then reload the HA :)

It‘s in the line 279 in that file. :-)

Hi, sorry I don´t know where the issue is - probably me. But same issue after restarting many times and also deleting and re installing integration:

Can you advise again please?

Did you save the file?