search

ollo69 / ha-melcloud-custom

Home Assistant custom integration for MelCloud AC devices with additional Swing control and WI-Fi Sensor
Apache License 2.0
21 stars 9 forks source link

Finished fetching melcloud_custom (success: False) #13

Open jburdy opened 8 months ago

jburdy commented 8 months ago

Hello, All my entities have been "unavailable" for a few hours. However, I can still control them from the iPhone app and the MELCloud website. I've activated the debug logs from the integration settings but that's all I get.

...
hass  | 2024-02-05 16:53:19.327 DEBUG (MainThread) [custom_components.melcloud_custom] Finished fetching melcloud_custom-Entree data in 0.235 seconds (success: False)
hass  | 2024-02-05 16:53:19.472 DEBUG (MainThread) [custom_components.melcloud_custom] Finished fetching melcloud_custom-Haut  data in 0.195 seconds (success: False)
hass  | 2024-02-05 16:53:19.496 DEBUG (MainThread) [custom_components.melcloud_custom] Finished fetching melcloud_custom-Carnotzet  data in 0.268 seconds (success: False)
...

How can I investigate further? Many thanks in advance,

mupfpilou commented 8 months ago

Same issue here :(

mupfpilou commented 8 months ago

Looks like Mitsubishi might have put a rate limit...

Error requesting melcloud_custom-x data: 429, message='This request has been throttled due to an excessive amount of traffic to our service.',

ollo69 commented 8 months ago

We have to wait a couple of days. Could be just related to server maintenance.

mupfpilou commented 8 months ago

Working via app though Hope it is only a temporary measure from Mitsu

jburdy commented 8 months ago

It's annoying that we have to go through their cloud. The devices respond in http (but require login/pass) isn't there a local API that can be used?

ollo69 commented 8 months ago

There is a project based on tasmota that require to connect an ESP Module replacing the Mitsubishi WiFi module. I started to implement this some times ago, but than I abandoned. If API cloud will become unusable I will come back to that solution.

https://isaiahchia.com/2022/06/16/integrating-mitsubishi-air-con-hvac-with-home-assistant-via-nodemcutasmota/

mupfpilou commented 8 months ago

Nothing found to discuss directly with the original wifi module ? It would be great to not replace all these expensive wifi modules :(

ollo69 commented 8 months ago

No as I know. I suppose that wifi module use proprietary language protocol to exchange data with Melcloud services.

limkinZero commented 8 months ago

You should use Ecodan local with a esp32 card. You will forget Melcloud forever.

https://github.com/rbroker/ecodan-ha-local

El lun, 5 feb 2024, 19:14, Julien Burdy @.***> escribió:

It's annoying that we have to go through their cloud. The devices respond in http (but require login/pass) isn't there a local API that can be used?

— Reply to this email directly, view it on GitHub https://github.com/ollo69/ha-melcloud-custom/issues/13#issuecomment-1927684923, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADLQJD4AR5IDXJQISIV2GI3YSEOO7AVCNFSM6AAAAABC2MNF42VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRXGY4DIOJSGM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

ollo69 commented 8 months ago

Just following native integration suggestion, I drastically reduce the polling frequency to once in 15 minutes in last release, hoping that it's enough. Honestly I think that this change will make this integration quite unusable and so I will probably move shortly to replace WiFi module with an ESP module. I don't think I'll miss Melcloud feature....

mupfpilou commented 8 months ago

Thx for the quick update I agree its not ideal but it is still worth something for people that invested in these wifi modules and not familiar with ESP stuff ! Personally, I have 5 units, and DIY ESPHome x5 cost some bucks, plus DIY work with cables / firmware / whatever ;)

mupfpilou commented 8 months ago

By the way, just tested 0.3.2, still unavailable sensors and theses logs on my 5 units

Error requesting melcloud_custom-xxx data: 429, message='This request has been throttled due to an excessive amount of traffic to our service.', url=URL('https://app.melcloud.com/Mitsubishi.Wifi.Client/EnergyCost/Report')
Error requesting melcloud_custom-xxx data: 429, message='This request has been throttled due to an excessive amount of traffic to our service.', url=URL('https://app.melcloud.com/Mitsubishi.Wifi.Client/EnergyCost/Report')
Error requesting melcloud_custom-xxx data: 429, message='This request has been throttled due to an excessive amount of traffic to our service.', url=URL('https://app.melcloud.com/Mitsubishi.Wifi.Client/EnergyCost/Report')
Error requesting melcloud_custom-xxx data: 429, message='This request has been throttled due to an excessive amount of traffic to our service.', url=URL('https://app.melcloud.com/Mitsubishi.Wifi.Client/EnergyCost/Report')
Error requesting melcloud_custom-xxx data: 429, message='This request has been throttled due to an excessive amount of traffic to our service.', url=URL('https://app.melcloud.com/Mitsubishi.Wifi.Client/EnergyCost/Report')

My account is no more unavailable via the app, but sensors are still not there Could it be related to this specific URL "/EnergyCost/Report" ?

I will try again tomorrow morning and will keep you informed

ollo69 commented 8 months ago

I suppose that we need to wait some times so that rate limit is removed, let's see what happen in next hours

jburdy commented 8 months ago

There's an http server in the wifi modules (mine's a MAC-577IF2-E). Can't we use it to control them locally? There really is something behind it, but I don't have the credentials.

Try this on one of your devices: http://192.168.1.189/apinfo

I'd rather hack the current modules than do DIY and throw them in the garbage can.

Grabauskas commented 8 months ago

Could it be that they've recently implemented rate limiting? I'm experiencing a similar issue in the app with additional details.

image

svmaris commented 8 months ago

Updated the plugin to 0.3.2 and things started working again for me a couple of hours ago.

keesedip commented 8 months ago

for me it's working again too.

ollo69 commented 8 months ago

There's an http server in the wifi modules (mine's a MAC-577IF2-E). Can't we use it to control them locally? There really is something behind it, but I don't have the credentials.

This is probably the web interface used to configure IP and wifi. To hack the wifi module you should capture and analyze the trafic between wifi module and melcloud (maybe wifi and your router is enough) that is protected by wifi security. Doesn't really seems so simple...

jburdy commented 8 months ago

Updated the plugin to 0.3.2 and things started working again for me a couple of hours ago.

But the question is: Is it working again because they've removed the API quotas, or is it working again thanks to @ollo69 change to the very slow 15min polling (thanks for reacting so quickly, it really helps).

ollo69 commented 8 months ago

But the question is: Is it working again because they've removed the API quotas, or is it working again thanks to @ollo69 change to the very slow 15min polling (thanks for reacting so quickly, it really helps).

Just install previous release and see what happen. I suppose that it work again due to very slow polling, the message from cloud service was clear 😉

Grabauskas commented 8 months ago

Now it's working with a 15-minute polling interval. Thanks for the fix!

There's an http server in the wifi modules (mine's a MAC-577IF2-E). Can't we use it to control them locally? There really is something behind it, but I don't have the credentials.

Try this on one of your devices: http://192.168.1.189/apinfo

I'd rather hack the current modules than do DIY and throw them in the garbage can.

@jburdy, I have the same wifi module, and I tried to find the solution. However, I discovered that it is quite complex. You can read this discussion here: https://github.com/ncaunt/meldec/issues/2

An alternative method exists using the EchonetLite protocol with echonetlite_homeassistant. But unfortunately, it is only supported in MAC-578, not MAC-577 😞

ollo69 commented 8 months ago

Unfortunately I think that the 8 at the end of the module model (MAC-xx8) means that implement the Echonet proto that is not implemented by MAC-xx7 sold in our region.

ollo69 commented 8 months ago

I just release a new version where is possible to configure the update interval via entry options, so you will be able to try to find the best supported value. The valid interval go from 60 to 1800 seconds, with a default of 900 seconds (15 minutes) as hardcoded in previous release.

swnl3 commented 8 months ago

A year ago I installed 13 Mitsubishi HVAC units. Melcloud integration was one of my main reasons. I got them fully automated through HA and node-red. They are controlled by solar production, door and window sensors and external temperature sensors.

Melcloud Custom made it even a little nicer with the added stuff.

Today I received an answer from Mitsubishi via the Melcloud app.

Vanwege toegang van ongeautoriseerde externe applicaties tot het MELCloud-platform heeft er op maandag 5 februari een beveiligingsupdate plaatsgevonden. Wanneer het MELCloud-platform onrechtmatig gebruik herkend zal het account voor een aantal uur worden geblokkeerd. Het account zal na enkele uren vanzelf weer gedeblokkeerd worden.

Due to access by unauthorized external applications to the MELCloud platform, a security update took place on Monday, February 5. If the MELCloud platform recognizes unlawful use, the account will be blocked for a number of hours. The account will be unblocked automatically after a few hours.

So external access is apparently unlawful and being actively blocked by Mitsubishi Electric rendering any other access then the Melcloud app or web useless. I read in the HA thread that it is in the TOS of ME that external apps are not allowed. Sadly I missed that when I bought these units.

I really hope someone will be able to find a way around this or figure out how to access the units locally. Any cloud controlled device will be a absolute no-go for me in the future..

ollo69 commented 8 months ago

Not so clear what is an unauthorized external applications. They have a web site, so any browser should be an external application, why not HA? And for sure it was authorized because we use our credentials as registered customer.

swnl3 commented 8 months ago

Not so clear what is an unauthorized external applications. They have a web site, so any browser should be an external application, why not HA? And for sure it was authorized because we use our credentials as registered customer.

I totally agree!

I am wondering what they try to get done or gain.

To me its a huge disappointment in the brand. I was happy with the performance of my units but by rendering them almost "dumb" again is.....

ToS: 3.7 You will: (a) only use the System and the Services via the Application or the Website;

10.3 We may terminate this Agreement: (a) at any time by providing thirty (30) days written notice by email to the address recorded on Your Service account or by other appropriate means; (b) with immediate effect if You are in breach of any the Agreement; and/or (c) in accordance with clauses 6.4 or 9.3.

mupfpilou commented 8 months ago

After having checked again what is going to their servers I really think the only issue is the fetch of energy reports at each 60s update. Browser is a client just like HA and is doing 60s xhr updates The 429 error is only related to the report endpoint It could be interesting to test without the call to reports in the underlying python lib

swnl3 commented 8 months ago

Would be great if Ollo69 could adapt the code making more selections of what to poll possible.

My experience is that the energy reporting feature sometimes calculates much more then what my local powerplugs measure so I replaced my energy monitoring with homewizard powerplugs which have a local api

ollo69 commented 8 months ago

I need to better check this in the library, but I think that energy report was fetched every 5 minutes (and now every 30 minutes). So I'm not convinced that this is the issue.

swnl3 commented 8 months ago

Maybe they also block at account level. I have not been able to get any of my accounts back to work. I updated and switched off polling. Also after hours (+8) I have not been able to get my HA reconnected.

ollo69 commented 8 months ago

Just released a new version that use new pymelcloud library to remove daily energy sensors and related polling. Please report here if this change eventually allow to use higher refresh rate.

swnl3 commented 8 months ago

Thank you for your quick response and effort!

Updating now and will follow-up asap.

So far I have not been able to reconnect. The melcloud app works but HA fails.

The native Melcloud integration was also updated with a lower polling rate however no joy for me at least.

Can anyone explain how they circumnavigated the block? New or guest accounts? I tried to setup a guest account and it immediately got blocked even before being able to activate my new account. I switched off polling in the integration yesterday.

ollo69 commented 7 months ago

Personally after installing integration version that use lower polling, it come back to work after some hours. I did no changes on my melcloud account.

swnl3 commented 7 months ago

For the time being I have been able to regain control via HA by making a guest account. Will update.

Thanks again for the quick follow-ups and updates!

I noticed on the Melcloud website that there is a authorized apps tab under the settings so apparently there is a way to interface. I don't recall setting this up but will report back when I figured it out

melcloud1

ollo69 commented 7 months ago

Just to let you know that I reduced my polling time to 600 sec (10 min.) and for now everything is working properly. I think this is also related to the number of devices polled, that in my case are 2.

svmaris commented 7 months ago

I've set it to 300s a couple of hours ago and it seems to work fine. I have 4 units. I'll keep an eye on it.

mupfpilou commented 7 months ago

I set it 2days ago to 600 with 5 units No pb so far Does this need to reboot HA to take effect ?

ollo69 commented 7 months ago

Does this need to reboot HA to take effect ?

No, change is immediate when you confirm the options. It reset the timer and start using the new value.

ddeconin-gh commented 7 months ago

I'm currently on MelCloud Custom v.0.3.3. (so still with the daily energy sensors). I'm on request timer of 800. 5 devices.

ollo69 commented 7 months ago

Could you make the daily energy consumption sensor disabled by default instead of removing it alltogether?

No, because there are no way to disable, this is controlled by the library pymecloud. In last release the code owner removed daily energy because seems to be the cause for disconnection.

ddeconin-gh commented 7 months ago

Could you make the daily energy consumption sensor disabled by default instead of removing it alltogether?

No, because there are no way to disable, this is controlled by the library pymecloud. In last release the code owner removed daily energy because seems to be the cause for disconnection.

Ok, got it, thx!

swnl3 commented 7 months ago

Sorry for the late reply.

I had to delete the integration, created guest accounts and wait for half a day. Now got it working again with 13 units!

It has been stable for the past days! Left the setting at 900s between polls. Haven't had time to experiment with lower values.

Thanks @ollo69

dragonbane0 commented 6 months ago

Kinda feared they would one day start to do this. Frustratingly echonet is supported on MAC-577 as well, but it can only be enabled by accessing the /config endpoint on the local port 80 webserver on the device. And for that you need the root/suser or admin password which we don't have. So any Mitsubishi service technician is probably able to enable this on the adapter, but seemingly it's currently policy to not do so