Site with several chargers

bjhemm commented 2 years ago

The problem

Not sure if this is an issue, support request or a misconfiguration by the site admins, but here goes..

The community where I live recently had all garages fitted with Equalizers and chargers. One Equalizer per garage and one charger per parking space. The integration installs just fine, I select the site and every charger gets its own device with alot of entities.

However, I can't control my charger. If I use the app and, for instance, lock the cable, I can see that it gets locked in Home Assistant. Trying to unlock in HA results in Forbidden service, 403. Is this a token issue?

2022-10-28 23:44:48.681 DEBUG (MainThread) [pyeasee.easee] POST: /api/chargers/XXXXXXXX/settings ({'json': {'lockCablePermanently': False}}) 2022-10-28 23:44:48.681 DEBUG (MainThread) [pyeasee.easee] verify_updated_token: 2022-10-29 23:43:39.723624, 2022-10-28 23:44:48.681834, False 2022-10-28 23:44:48.747 ERROR (MainThread) [pyeasee.easee] Forbidden service (403: {'type': 'https://tools.ietf.org/html/rfc7231#section-6.5.3', 'title': 'Forbidden', 'stas': 403, 'traceId': '00-6ee9ce760792cc6172895f53ebd57da3-4faa5781f6b2d748-00'} https://api.easee.cloud/api/chargers/XXXXXXXX/settings)

Version of Easee integration having the issue?

0.9.46

Version of Home Assistant Core having the issue?

Home Assistant 2022.10.5

Anything in the logs that might be useful for us?

No response

Additional information

No response

olalid commented 2 years ago

You are using the same login in the app and HA, right?

bjhemm commented 2 years ago

Yes, same user/pw in the app and integration.

I see the logs getting flooded by 401s against the Equalizer, not sure if this is related as I am trying to control my charger. I am assuming I would see a similar token request for my charger, which I'm not.

2022-10-29 22:46:03.895 DEBUG (MainThread) [pyeasee.easee] GET: /api/equalizers/XXXXXXXX/state ({}) 2022-10-29 22:46:03.895 DEBUG (MainThread) [pyeasee.easee] verify_updated_token: 2022-10-30 22:44:44.262220, 2022-10-29 22:46:03.895297, False 2022-10-29 22:46:04.153 DEBUG (MainThread) [pyeasee.easee] Unautorized (401: {'message': 'Unauthorized'} https://api.easee.cloud/api/equalizers/XXXXXXXX/state) 2022-10-29 22:46:04.153 DEBUG (MainThread) [pyeasee.easee] Re authorizing due to 401 2022-10-29 22:46:04.153 DEBUG (MainThread) [pyeasee.easee] getting token for user: xxxx@xxxx.xxx 2022-10-29 22:46:04.279 DEBUG (MainThread) [pyeasee.easee] TOKEN: {'accessToken': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImN0eSI6IkpXVCJ9.eyJBY2NvdW50SWQiOjYwMTIzMiwiVXNlcklkIjo1NTg0MDgsInVuaXF1ZV9uYW1lIjoiQmrDtnJuIEhlbW1pbmdzc29uICIsInJvbGUiOiJVc2VyIiwibmJmIjoxNjY3MDc2MzY0LCJleHAiOjE2NjcxNjI3NjQsImlhdCI6MTY2NzA3NjM2NH0.Nhef3tj2d1mqYY5fEzU1N1l3pAT-u4b-lw-WS5pzpbI', 'expiresIn': 86400, 'accessClaims': ['User'], 'tokenType': 'Bearer', 'refreshToken': 'NQWUvl6apDC+7umVeq4eO39u5kD+Eoa44GSrXTtX0v4='} 2022-10-29 22:46:04.279 DEBUG (MainThread) [pyeasee.easee] Unautorized (401: {'message': 'Unauthorized'} https://api.easee.cloud/api/equalizers/XXXXXXXX/state)

If you need any more info/logs, let me know. :)

olalid commented 2 years ago

All log message that says something like this: "2022-10-28 23:44:48.681 DEBUG (MainThread) [pyeasee.easee] verify_updated_token: 2022-10-29 23:43:39.723624, 2022-10-28 23:44:48.681834, False" is just telling you that the token expiration time was checked and that it was concluded that it has not expired. (False = not expired).

In your second log, you can however see that the token is updated, but that is because of the 401 response which triggers the HA integration to try to log in again. Note that the accessToken in the log you pasted here above will allow anyone to access your account for 24 hours. So if you are still using that token, you should restart the integration to make sure it gets invalidated.

I do not see anything here that point to a problem with the token handling in any case. To me it seems the API just does not think you are allowed to access the specific endpoints we are using in the integration. We have not really tested the integration with larger installations like this, so this is unknown territory for us.

in the first case you mention with the cable lock it might just be so that the API we are using is not working in installations like these, there is another API specifically for this setting that we could try using instead. I.e. https://developer.easee.cloud/reference/post_api-chargers-id-commands-lock-state

Regarding that second case: do you see the equalizer and its values in the Easee app?

The serial number in the API call above (2022-10-29 22:46:03.895 DEBUG (MainThread) [pyeasee.easee] GET: /api/equalizers/XXXXXXXX/state ({})" is that XXXXX matching your equalizer serial number?

Are you also seeing all your neighbours devices in both the app and in the integration?

bjhemm commented 2 years ago

Yes, I do a reload everytime I send something. Thanks for noticing it though. :)

I'd be happy to help getting large-ish installations to work. I think there are 82 chargers in total, divided in several garages, and I GUESS, one Equalizer per garage.

Regarding the 403, is there any way I can test the other API in HA?

Regarding the 401 and serial number, I assume it is correct, I have no real way to verify other than I get the same number everytime I reinstall the integration, or query the API myself. I see no real details, just names like "import_power", "export_power".. etc.

Neighbour devices/Equalizer. No, they are not visible in the app(only my charger visible). In the HA integration all are visible though.

Edit: In fact I remember I was playing around yesterday with the exact API you mentioned at https://developer.easee.cloud/reference/post_api-chargers-id-commands-lock-state That actually worked. It seems the integration uses this? https://developer.easee.cloud/reference/post_api-chargers-id-settings If I try this at the "API-site", I get the same 403 forbidden.

olalid commented 2 years ago

OK, so that points to that that we could solve that first problem by changing the API we use, and it would not cause any problems for anyone else most likely. Any place where we use the settings API will probably have the same issue, so we should look in to that. This is actually a change in the pyeasee lib, which makes it a bit more difficult for you to test. But if you can find the lib source files on your hard drive, you can edit them just as any other file.

But do you see a single equalizer in HA or one for each charger? If you see only one, then there is probably only one for the entire installation and you probably do not have access to it. It is the "normal" case that there is one for each installation, since its purpose is to monitor the loading of the entire installations main fuses.

In any case it seems the app somehow knows which charger is yours, and that it should not care about the rest... Right now I do not know where to find that information. Apart from looking at the 401/403, but that seems like an odd solution...

olalid commented 2 years ago

If you have a separate computer where you have python installed and can make some tests, you could try to install the pyeasee library (pip install pyeasee) and then run: "pyeasee -u <username> -p <password> -s". This should give you the output that we are using to find all the chargers and equalisers. (username = +<countrycode><phone number>). If you can see anything that is different between the charger that is yours and the ones that are not yours it would be good to know...

bjhemm commented 2 years ago

I see one Equalizer in HA, and I only see the chargers that are in my garage. Every garage is its own site, named Garage-A, Garage-B etc.

Ok, ran pyeasee.. At first glance, there's not much that differs between the chargers apart from creation date and masterBackPlateId, levelOfAccess is set to 1 on my charger and one other charger, levelOfAccess is set to null on the others. Just guessing, but this might be because there are not that many EVs around here.. yet, and that only me and someone else has "activated" the charger.

astrandb commented 2 years ago

Please do a diagnostic download from the the Easee integration card and upload it to this issue. We should then be able to see how the sites are laid out. At least what data that can be read with your account.

astrandb commented 2 years ago

There is an API endpoint /api/accounts/products that likely will list the products accessible by the account. I cannot test it now since the Easee cloud is having big problems.

bjhemm commented 2 years ago

Here you go, the diag file. config_entry-easee-2cae9e1fc8efa428d93e93b559c9931d.json.txt

Also tried /api/accounts/products, and it is as you say, it only list my charger.

astrandb commented 2 years ago

@bjhemm Thank you for the info. It looks like we have to re-design the initial retrieval of sites, circuits and chargers to be compatible with this type of multi-user installation. There is no obvious reason why the POSTs fail but we will dig into it.

bjhemm commented 2 years ago

Great. Please let me know if you need more info/any testing done.

astrandb commented 2 years ago

@bjhemm We have impleneted a fix that should solve your issue. However, we cannot verify it fully since we don't have access to a large installation like yours. So please remove your current installation of the integration from HA. Then reinstall version "master" in HACS, restart HA and reinstall the integration. If everything works you should only see the products that your account give access to. The permanent cable lock switch should work. But, there are a lot of services in the integration that applies to installations where you have complete access to all products and functions. If you try them you should get a 403-Forbidden error but no other harm.

We appreciate your feedback.

bjhemm commented 2 years ago

I still see all products. If i interpret the log correctly, /api/accounts/products find the "site id", then /api/sites/"site id" get all the details of that site including the other chargers. This is more of an "annoyance" than a problem(for me at least). I can always disable the other chargers so they don't show in my entities list. :)

Good news, cable lock works as intended. Just wondering about the other settings, Enable idle current and Smart charging, what are they? Are they even useful in an installation like this?

astrandb commented 2 years ago

I still see all products

OK, I understand why. We should try to filter the unaccessible chargers out of sight to reduce annoyance and confusion.

Enable idle current and Smart charging

Those settings use the same type of API as cable lock and should work. Enable Idle current is used to reserve 6A for a charger even if other chargers on the same circuit asks for all available current. Smart charging is basically only changing LED color from white to blue, but can be used by an operator to trigger some actions. I don't know how these settings are implemented in a multi-account installation as yours.

We will be back with a new attempt on first item.

bjhemm commented 2 years ago

I think Ola changed the lock cable API from settings to commands. See below. Anyway, I don't know if they are needed or not, just posting this for "documentation purposes". :)

2022-10-31 22:53:57.888 DEBUG (MainThread) [pyeasee.easee] POST: /api/chargers/XXXXXXXX/commands/lock_state ({'json': {'state': True}}) 2022-10-31 22:53:59.458 DEBUG (MainThread) [custom_components.easee.controller] Callback XXXXXXXX 30 state_lockCablePermanently True 2022-10-31 22:54:03.466 DEBUG (MainThread) [custom_components.easee.controller] Callback XXXXXXXX 103 state_cableLocked True 2022-10-31 22:54:04.186 DEBUG (MainThread) [pyeasee.easee] POST: /api/chargers/XXXXXXXX/settings ({'json': {'enableIdleCurrent': True}}) 2022-10-31 22:54:04.249 ERROR (MainThread) [pyeasee.easee] Forbidden service (403: {'type': 'https://tools.ietf.org/html/rfc7231#section-6.5.3', 'title': 'Forbidden', 'status': 403, 'traceId': '00-fa40590dbc80ff27eb03852f114eef3a-99a4de86e1dffc0e-00'} https://api.easee.cloud/api/chargers/XXXXXXXX/settings) 2022-10-31 22:54:09.737 DEBUG (MainThread) [pyeasee.easee] POST: /api/chargers/XXXXXXXX/settings ({'json': {'smartCharging': True}}) 2022-10-31 22:54:09.800 ERROR (MainThread) [pyeasee.easee] Forbidden service (403: {'type': 'https://tools.ietf.org/html/rfc7231#section-6.5.3', 'title': 'Forbidden', 'status': 403, 'traceId': '00-1feca031b0c0068c9fcf4c1adb14ffee-4500064b1c52479c-00'} https://api.easee.cloud/api/chargers/XXXXXXXX/settings)

Tack för insatsen så här långt. :)

astrandb commented 2 years ago

My bad. I will have a look if there are alternate API endpoints for these functions. Otherwise we must accept that they are forbidden in the context if this installation type.

olalid commented 2 years ago

There is an API for smart charging (https://developer.easee.cloud/reference/post_api-chargers-id-commands-smart-charging). However, strangely its syntax does not allow you to disable, only enable, and to disable you have to use the settings API, which does not work. Unless it works when you change it to False, but that seems unlikely...

olalid commented 2 years ago

When I made changes to the pyeasee lib first time I did not realise it calls another function to retrieve more detailed information about the site. This screws things up because it overwrites the information about what products you have access to. I have written another PR now that should solve that.

astrandb commented 2 years ago

@bjhemm It is difficult for us to test this fix. As we are owners of our sites we have full access to all products and services. We would appreciate a complete output from the /api/accounts/products endpoint so we can make sure that we pick up the correct products when initializing the integration. If you dont want to leave the data here you can mail it to me at ake@strandberg.eu

olalid commented 2 years ago

@bjhemm could you please test with the 0.7.47rc1 version of the lib to see if that removes all of the chargers you do not have access to? You can do that by editing the manifest.json file in your custom_components/easee folder such that it says "pyeasee==0.7.47rc1" and then restart HA. (Yes, I know I kind of screwed up the version numbers, it should have been 0.7.48rc1).

bjhemm commented 2 years ago

Ok, I installed the latest master and edited the file and restarted. I still see every charger.

Ola, kan du maila mig? Har några grejer jag inte vill yppa öppet här. bjhemm[.at.]gmail.com.

erik-bjork commented 2 years ago

I have the same problem as @bjhemm. I see all chargers in my garage (73) and I have no control over my own. All entities and controls are unavailable.

Tried editing manifest.json but no change.

astrandb commented 2 years ago

@erik-bjork We are working with this issue and there are a few significant problems left to be handled. Here is the way to get the latest development version which will hopefullt remove all but one of the chargers:

Remove the Easee integration from HA
You need to reinstall the Easee integration in HACS - select version "master"
Restart HA
Edit custom_components/easee/manifest.json -> pyeasee==0.7.47rc1
Install the Easee integration again.

erik-bjork commented 2 years ago

@erik-bjork We are working with this issue and there are a few significant problems left to be handled. Here is the way to get the latest development version which will hopefullt remove all but one of the chargers:

Remove the Easee integration from HA

You need to reinstall the Easee integration in HACS - select version "master"

Restart HA

Edit custom_components/easee/manifest.json -> pyeasee==0.7.47rc1

Install the Easee integration again.

Just did that, no change unfortunately.

astrandb commented 2 years ago

The integration was designed for typical home installation where you have owner or admin privileges so there is need for a few workarounds to adapt to the user privilege level. Please get a diagnostic download from the integration card and upload it to this issue.

erik-bjork commented 2 years ago

easee log.txt

Here you go.

"P-plass 93" is mine.

astrandb commented 2 years ago

Odd... Are you sure that you did:

You need to reinstall the Easee integration in HACS - select version "master"

Restart HA

Please try once again. Note that you need to remove the installation from Home Assistant and add it back to clean out all stale chargers.

olalid commented 2 years ago

I should clarify that it did work for bjhemm later, it is just not visible here because we had that conversation in mail.

As astranb says, it is important to follow all of the steps he listed, and you need to do them in that order, if you edit manifast.json first, and then later download the master version of HA it will overwrite your change. Also, you may need to do an extra restart afterwards because HA sometimes remembers old entities.

bjhemm commented 2 years ago

@erik-bjork. It didn't work for me at first either. I redid all steps a few hours later. From then on I only see my charger. :)

erik-bjork commented 2 years ago

So, I re-did all the steps above. I am positive I did them in the correct order. Still, no change. I get all the chargers and have no control over my own (or any other for that matter). Anything else I can give you?

I checked the logs and I get spammed by errors: 2022-11-03 18:53:22.192 ERROR (MainThread) [pyeasee.easee] Forbidden service (403: {'type': 'https://tools.ietf.org/html/rfc7231#section-6.5.3', 'title': 'Forbidden', 'status': 403, 'traceId': '00-587a0d747b184478f67d56e5bee883a9-71334244109d9755-00'} https://api.easee.cloud/api/sites/[5 digits]/state)

and

2022-11-03 18:53:22.464 ERROR (MainThread) [pyeasee.easee] Forbidden service (403: {'message': 'Forbidden'} https://api.easee.cloud/api/chargers/[5 digits]/state)

astrandb commented 2 years ago

OK, then we must drill deeper.

Please goto the Easee developer portal Products (easee.cloud)https://developer.easee.cloud/reference/get_api-accounts-products. Login with your username and password (same as in the app) and execute the command /api/accounts/products. You can then send the output to me @.**@.> if you don’t want to disclose the id:s etc on this site.

Från: erik-bjork @.> Skickat: den 3 november 2022 19:00 Till: fondberg/easee_hass @.> Kopia: Åke Strandberg @.>; State change @.> Ämne: Re: [fondberg/easee_hass] Site with several chargers (Issue #239)

So, I re-did all the steps above. I am positive I did them in the correct order. Still, no change. I get all the chargers and have no control over my own (or any other for that matter). Anything else I can give you?

I checked the logs and I get spammed by errors: 2022-11-03 18:53:22.192 ERROR (MainThread) [pyeasee.easee] Forbidden service (403: {'type': 'https://tools.ietf.org/html/rfc7231#section-6.5.3', 'title': 'Forbidden', 'status': 403, 'traceId': '00-587a0d747b184478f67d56e5bee883a9-71334244109d9755-00'} https://api.easee.cloud/api/sites/[5 https://api.easee.cloud/api/sites/%5b5 digits]/state)

and

2022-11-03 18:53:22.464 ERROR (MainThread) [pyeasee.easee] Forbidden service (403: {'message': 'Forbidden'} https://api.easee.cloud/api/chargers/[5 https://api.easee.cloud/api/chargers/%5b5 digits]/state)

— Reply to this email directly, view it on GitHubhttps://github.com/fondberg/easee_hass/issues/239#issuecomment-1302480872, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABGITSVHZOOEVFFA25JI323WGP4T3ANCNFSM6AAAAAARROK6KA. You are receiving this because you modified the open/close state.Message ID: @.**@.>>

erik-bjork commented 2 years ago

Looks like your email was hidden

astrandb commented 2 years ago

One more try. ake@strandberg.eu

olalid commented 2 years ago

Sorry, I deleted my comment since it was wrong and irrelevant (in case you already read it).

olalid commented 2 years ago

Can you verify the version of the lib in the log, there should be a line like this during startup of HA: 2022-11-02 22:35:00.006 INFO (MainThread) [pyeasee.easee] Easee python library version: 0.7.47rc1

erik-bjork commented 2 years ago

@astrandb Check your email :)

@olalid Changed the logging level to INFO and restarted, but the only mention of easee I get in the logs are the standard custom integration warning... I am, however, certain that manifest.json has been edited correctly.

astrandb commented 2 years ago

Thanks, there is now an updated test version of the integration available. Please follow the (too?) detailed instructions

Remove the Easee integration from settings->integrations
Restart HA and check that the integration and all devices and entities belonging to Easee integration are gone.
Goto HACS->Integrations, Click on the Easee card
Select Redownload from the overflow menu
Select version "master" and download
Back out and restart HA
Go into HACS->Integrations->Easee and verify that you have version 3da1423 in upper left corner
Go to Settings->Integrations and add the Easee integration.

Hope this will work now so that you only see the charger(s) that you have access to.

astrandb commented 2 years ago

@erik-bjork To enable full debug logging for this integration you can enter following to configuration.yaml

logger:
  default: info
  logs:
    pyeasee: debug
    custom_components.easee: debug

erik-bjork commented 2 years ago

Thanks, there is now an updated test version of the integration available. Please follow the (too?) detailed instructions

Remove the Easee integration from settings->integrations

Restart HA and check that the integration and all devices and entities belonging to Easee integration are gone.

Goto HACS->Integrations, Click on the Easee card

Select Redownload from the overflow menu

Select version "master" and download

Back out and restart HA

Go into HACS->Integrations->Easee and verify that you have version 3da1423 in upper left corner

Go to Settings->Integrations and add the Easee integration.

Hope this will work now so that you only see the charger(s) that you have access to.

The version of the Easee integration in HACS says 0.9.46 in the upper corner, but HACS tells me to update it from 3da1423 from the main page so it's probably correct.

But anyway, it works now! Only get my own charger, and I can control it as expected plus all the sensors work. Great job!

astrandb commented 1 year ago

Completed

nordicopen / easee_hass