New libbi action `chargefromgrid`

[mattburns]

refs issue #15

This change allows you to toggle charging from grid. Toggle on:

python cli.py libbi chargefromgrid true

Disable charging from grid:

python cli.py libbi chargefromgrid false

I won't be offended if you want to re-write my implementation, it just proves the mechanism.

For background: These actions happen over a different api endpoint that uses AWS Cognito for OAuth authentication.

A simple demonstration of how to interact with this api is like this:

  import requests
  from pycognito import Cognito

  u = Cognito('eu-west-2_E57cCJB20','2fup0dhufn5vurmprjkj599041',
      username='youremail')

  u.authenticate(password='yourpassword')

  headers = {"Authorization": f"Bearer {u.access_token}"}
  print(requests.get("https://myaccount.myenergi.com/api/Product/UserHubsAndDevices", headers=headers).json())
  print(requests.put("https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?chargeFromGrid=true&serialNo=yourserial", headers=headers).json())

[videojedi]

How could the authentication work, if an account is using one of the other methods myenergi support? I'm using 'sign in with apple' for example. Also google and facebook are offered.

[trizmark]

Once this is merged I have more stuff to add. Got a number of useful endpoints from the Android app...

[FlipGFlop]

@videojedi - my curl test script uses the serial number of the vhub (my libbi) as the user and the APIKey as the password. This is available from via the 'Advanced' button on the location products page from the web UI, https://myaccount.myenergi.com/location#products.

[mattburns]

@FlipGFlop , this project has arguments for username and password which are being used as the serial and apikey. I assume I'm missing something so rather than refactor something I didn't understand, and introduce a breaking change, this PR adds 2 new arguments: app_email and app_paasword to represent the email auth credentials you login with on the app/website.

Perhaps they should be renamed while keeping the old names as aliases for backwards compatibility?

[videojedi]

Sorry, I'm getting an exception when I try to run....

vscode ➜ / $ myenergi libbi chargefromgrid false Traceback (most recent call last): File "/home/vscode/.local/bin/myenergi", line 33, in sys.exit(load_entry_point('pymyenergi', 'console_scripts', 'myenergi')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 238, in cli loop.run_until_complete(main(args)) File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 100, in main await device.set_charge_from_grid(args.arg[0]) File "/home/vscode/pymyenergi/pymyenergi/libbi.py", line 184, in set_charge_from_grid await self._connection.put( File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 120, in put return await self.send("PUT", url, data, oauth) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 75, in send raise MyenergiException(response.status_code) pymyenergi.exceptions.MyenergiException

[mattburns]

No need to say sorry, will def be my fault, thanks for the feedback! Do you get the same error if you run it like this?

python cli.py libbi chargefromgrid false

I think you can pass -d for debug logging...

[trizmark]

@FlipGFlop This specific API endpoint uses OAuth, which requires the app username and password. I have just tested it and it does not work with serial number + API key. The serial + API key combo works for API endpoints that use digest auth. Those endpoints are served by sX.myenergi.net.

[trizmark]

@mattburns I am trying to test your fork, but running into issues. I am getting botocore.errorfactory.NotAuthorizedException: An error occurred (NotAuthorizedException) when calling the RespondToAuthChallenge operation: Incorrect username or password.

I have double-checked both and they are correct. I have copied your example cognito test from above and have been successfully using various API endpoints. I will do some more testing/debugging later this afternoon and report back.

[trizmark]

Ok, the auth error was due to me having special characters in my password.

I can confirm that enabling/disabling charging works fine! Excellent work! If I may have a suggestion: could we change the parameters 'true/false' to 'enable/disable'? They would less 'coder-y' and would be more in line with what you see on the app UI (Enable charging from the grid).

[videojedi]

tried again, removed special characters from password, but still fails for me.

vscode ➜ ~/pymyenergi (main) $ python cli.py -d libbi chargefromgrid false

DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'PUT']> DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 400, b'Bad Request', [(b'Content-Type', b'application/json; charset=utf-8'), (b'Content-Length', b'92'), (b'Connection', b'keep-alive'), (b'Date', b'Thu, 28 Sep 2023 13:43:58 GMT'), (b'Server', b'Kestrel'), (b'Cache-Control', b'no-cache,no-store'), (b'Expires', b'-1'), (b'Pragma', b'no-cache'), (b'X-Cache', b'Error from cloudfront'), (b'Via', b'1.1 9a9edb00220c3ef50c1919f84fea4888.cloudfront.net (CloudFront)'), (b'X-Amz-Cf-Pop', b'LHR61-P1'), (b'Alt-Svc', b'h3=":443"; ma=86400'), (b'X-Amz-Cf-Id', b'f-2FuEB4ivEcvyJaO4KDcCpa5X1FVq2TjlQkXrW9ntTh1wu7IhD5QA==')]) INFO:httpx:HTTP Request: PUT https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?chargeFromGrid=false&serialNo=xxxxxxxx "HTTP/1.1 400 Bad Request" DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'PUT']> DEBUG:httpcore.http11:receive_response_body.complete DEBUG:httpcore.http11:response_closed.started DEBUG:httpcore.http11:response_closed.complete DEBUG:pymyenergi.connection:PUT status 400 DEBUG:httpcore.connection:close.started DEBUG:httpcore.connection:close.complete Traceback (most recent call last): File "/home/vscode/pymyenergi/cli.py", line 4, in cli() File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 238, in cli loop.run_until_complete(main(args)) File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 100, in main await device.set_charge_from_grid(args.arg[0]) File "/home/vscode/pymyenergi/pymyenergi/libbi.py", line 184, in set_charge_from_grid await self._connection.put( File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 120, in put return await self.send("PUT", url, data, oauth) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 75, in send raise MyenergiException(response.status_code) pymyenergi.exceptions.MyenergiException

[trizmark]

If it was a username/password issue, then you'd get a different exception. Authentication is done on lines 39-40 of connection.py self.oauth = Cognito(_USER_POOL_ID, _CLIENT_ID, username=self.app_email) self.oauth.authenticate(password=self.app_password)

Silly question: your debug log shows your S/N as xxxxxxxxx - did you remove it before you posted the log? Could you post some of the previous lines leading up to the exception?

[videojedi]

Heres the complete log.. (yes I did remove serial number, not sure if thats strictly necessary...)

vscode ➜ ~/pymyenergi (main) $ python cli.py -d libbi chargefromgrid false DEBUG:pymyenergi.client:Refreshing data for all myenergi devices DEBUG:httpx:load_ssl_context verify=True cert=None trust_env=True http2=False DEBUG:httpx:load_verify_locations cafile='/home/vscode/.local/lib/python3.11/site-packages/certifi/cacert.pem' DEBUG:pymyenergi.connection:Get Myenergi base url from director DEBUG:httpcore.connection:connect_tcp.started host='director.myenergi.net' port=443 local_address=None timeout=20 socket_options=None DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff98803c10> DEBUG:httpcore.connection:start_tls.started ssl_context=<ssl.SSLContext object at 0xffff98026450> server_hostname='director.myenergi.net' timeout=20 DEBUG:httpcore.connection:start_tls.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff98663710> DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'GET']> DEBUG:httpcore.http11:send_request_headers.complete DEBUG:httpcore.http11:send_request_body.started request=<Request [b'GET']> DEBUG:httpcore.http11:send_request_body.complete DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'GET']> DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 401, b'Unauthorized', [(b'Date', b'Thu, 28 Sep 2023 13:43:57 GMT'), (b'Content-Length', b'0'), (b'Connection', b'keep-alive'), (b'X-Content-Type-Options', b'nosniff'), (b'X-Frame-Options', b'SAMEORIGIN'), (b'X-XSS-Protection', b'1; mode=block'), (b'Strict-Transport-Security', b'max-age=31536000; includeSubDomains'), (b'WWW-Authenticate', b'Digest realm="MyEnergi Telemetry", nonce="zMY2n0GYLVYVBLcHD5KH0G4zQaGT21d5", opaque="78b66b1db9dc49dd845535e7d7b35287", algorithm=MD5, qop="auth"'), (b'x_myenergi-asn', b'undefined'), (b'Access-Control-Allow-Credentials', b'true'), (b'Access-Control-Allow-Headers', b'Origin, Content-Type, Accept, Cookie'), (b'Access-Control-Allow-Origin', b'https://admin-ui.s18.myenergi.net')]) INFO:httpx:HTTP Request: GET https://director.myenergi.net/cgi-jstatus-E "HTTP/1.1 401 Unauthorized" DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'GET']> DEBUG:httpcore.http11:receive_response_body.complete DEBUG:httpcore.http11:response_closed.started DEBUG:httpcore.http11:response_closed.complete DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'GET']> DEBUG:httpcore.http11:send_request_headers.complete DEBUG:httpcore.http11:send_request_body.started request=<Request [b'GET']> DEBUG:httpcore.http11:send_request_body.complete DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'GET']> DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Date', b'Thu, 28 Sep 2023 13:43:57 GMT'), (b'Content-Type', b'application/json; charset=utf-8'), (b'Content-Length', b'538'), (b'Connection', b'keep-alive'), (b'X-Content-Type-Options', b'nosniff'), (b'X-Frame-Options', b'SAMEORIGIN'), (b'X-XSS-Protection', b'1; mode=block'), (b'Strict-Transport-Security', b'max-age=31536000; includeSubDomains'), (b'x_myenergi-asn', b's18.myenergi.net'), (b'Access-Control-Allow-Credentials', b'true'), (b'Access-Control-Allow-Headers', b'Origin, Content-Type, Accept, Cookie'), (b'Access-Control-Allow-Origin', b'https://admin-ui.s18.myenergi.net'), (b'ETag', b'W/"21a-MHybMKcsLiQv/57hZtBM8s4Nocw"')]) INFO:httpx:HTTP Request: GET https://director.myenergi.net/cgi-jstatus-E "HTTP/1.1 200 OK" DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'GET']> DEBUG:httpcore.http11:receive_response_body.complete DEBUG:httpcore.http11:response_closed.started DEBUG:httpcore.http11:response_closed.complete INFO:pymyenergi.connection:Updated myenergi active server to https://s18.myenergi.net DEBUG:pymyenergi.connection:GET /cgi-get-app-key- https://s18.myenergi.net/cgi-get-app-key- DEBUG:httpcore.connection:connect_tcp.started host='s18.myenergi.net' port=443 local_address=None timeout=20 socket_options=None DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff98603b90> DEBUG:httpcore.connection:start_tls.started ssl_context=<ssl.SSLContext object at 0xffff98026450> server_hostname='s18.myenergi.net' timeout=20 DEBUG:httpcore.connection:start_tls.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff98603a50> DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'GET']> DEBUG:httpcore.http11:send_request_headers.complete DEBUG:httpcore.http11:send_request_body.started request=<Request [b'GET']> DEBUG:httpcore.http11:send_request_body.complete DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'GET']> DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Date', b'Thu, 28 Sep 2023 13:43:57 GMT'), (b'Content-Type', b'application/json; charset=utf-8'), (b'Content-Length', b'1432'), (b'Connection', b'keep-alive'), (b'X-Content-Type-Options', b'nosniff'), (b'X-Frame-Options', b'SAMEORIGIN'), (b'X-XSS-Protection', b'1; mode=block'), (b'Strict-Transport-Security', b'max-age=31536000; includeSubDomains'), (b'x_myenergi-asn', b's18.myenergi.net'), (b'Access-Control-Allow-Credentials', b'true'), (b'Access-Control-Allow-Headers', b'Origin, Content-Type, Accept, Cookie'), (b'Access-Control-Allow-Origin', b'https://admin-ui.s18.myenergi.net'), (b'ETag', b'W/"598-ICKKbRcF73eWSkv7kSqGK7wrmhI"')]) INFO:httpx:HTTP Request: GET https://s18.myenergi.net/cgi-get-app-key- "HTTP/1.1 200 OK" DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'GET']> DEBUG:httpcore.http11:receive_response_body.complete DEBUG:httpcore.http11:response_closed.started DEBUG:httpcore.http11:response_closed.complete DEBUG:pymyenergi.connection:GET status 200 DEBUG:httpcore.connection:close.started DEBUG:httpcore.connection:close.complete DEBUG:httpcore.connection:close.started DEBUG:httpcore.connection:close.complete DEBUG:httpx:load_ssl_context verify=True cert=None trust_env=True http2=False DEBUG:httpx:load_verify_locations cafile='/home/vscode/.local/lib/python3.11/site-packages/certifi/cacert.pem' DEBUG:pymyenergi.connection:GET /cgi-jstatus- https://s18.myenergi.net/cgi-jstatus- DEBUG:httpcore.connection:connect_tcp.started host='s18.myenergi.net' port=443 local_address=None timeout=20 socket_options=None DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff9860dad0> DEBUG:httpcore.connection:start_tls.started ssl_context=<ssl.SSLContext object at 0xffff98026d50> server_hostname='s18.myenergi.net' timeout=20 DEBUG:httpcore.connection:start_tls.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff9860da10> DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'GET']> DEBUG:httpcore.http11:send_request_headers.complete DEBUG:httpcore.http11:send_request_body.started request=<Request [b'GET']> DEBUG:httpcore.http11:send_request_body.complete DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'GET']> DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Date', b'Thu, 28 Sep 2023 13:43:58 GMT'), (b'Content-Type', b'application/json; charset=utf-8'), (b'Content-Length', b'2616'), (b'Connection', b'keep-alive'), (b'X-Content-Type-Options', b'nosniff'), (b'X-Frame-Options', b'SAMEORIGIN'), (b'X-XSS-Protection', b'1; mode=block'), (b'Strict-Transport-Security', b'max-age=31536000; includeSubDomains'), (b'x_myenergi-asn', b's18.myenergi.net'), (b'Access-Control-Allow-Credentials', b'true'), (b'Access-Control-Allow-Headers', b'Origin, Content-Type, Accept, Cookie'), (b'Access-Control-Allow-Origin', b'https://admin-ui.s18.myenergi.net'), (b'ETag', b'W/"a38-Z1HoaV0nkKzsnJcGuZ1sr2C8ZXk"')]) INFO:httpx:HTTP Request: GET https://s18.myenergi.net/cgi-jstatus-* "HTTP/1.1 200 OK" DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'GET']> DEBUG:httpcore.http11:receive_response_body.complete DEBUG:httpcore.http11:response_closed.started DEBUG:httpcore.http11:response_closed.complete DEBUG:pymyenergi.connection:GET status 200 DEBUG:httpcore.connection:close.started DEBUG:httpcore.connection:close.complete DEBUG:pymyenergi.client:Adding eddi eddi-XXXXXXXX DEBUG:pymyenergi.client:Adding zappi Zappi Gate DEBUG:pymyenergi.client:Adding zappi Zappi Door DEBUG:pymyenergi.client:Adding harvi harvi-XXXXXXXX DEBUG:pymyenergi.client:Adding libbi libbi-XXXXXXXX DEBUG:pymyenergi.client:Unknown device type: asn DEBUG:httpx:load_ssl_context verify=True cert=None trust_env=True http2=False DEBUG:httpx:load_verify_locations cafile='/home/vscode/.local/lib/python3.11/site-packages/certifi/cacert.pem' DEBUG:pymyenergi.connection:PUT /api/AccountAccess/LibbiMode?chargeFromGrid=false&serialNo=XXXXXXXX https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?chargeFromGrid=false&serialNo=XXXXXXXX DEBUG:httpcore.connection:connect_tcp.started host='myaccount.myenergi.com' port=443 local_address=None timeout=20 socket_options=None DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff9860a110> DEBUG:httpcore.connection:start_tls.started ssl_context=<ssl.SSLContext object at 0xffff98026600> server_hostname='myaccount.myenergi.com' timeout=20 DEBUG:httpcore.connection:start_tls.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff9860a050> DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'PUT']> DEBUG:httpcore.http11:send_request_headers.complete DEBUG:httpcore.http11:send_request_body.started request=<Request [b'PUT']> DEBUG:httpcore.http11:send_request_body.complete DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'PUT']> DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 400, b'Bad Request', [(b'Content-Type', b'application/json; charset=utf-8'), (b'Content-Length', b'92'), (b'Connection', b'keep-alive'), (b'Date', b'Thu, 28 Sep 2023 13:43:58 GMT'), (b'Server', b'Kestrel'), (b'Cache-Control', b'no-cache,no-store'), (b'Expires', b'-1'), (b'Pragma', b'no-cache'), (b'X-Cache', b'Error from cloudfront'), (b'Via', b'1.1 9a9edb00220c3ef50c1919f84fea4888.cloudfront.net (CloudFront)'), (b'X-Amz-Cf-Pop', b'LHR61-P1'), (b'Alt-Svc', b'h3=":443"; ma=86400'), (b'X-Amz-Cf-Id', b'f-2FuEB4ivEcvyJaO4KDcCpa5X1FVq2TjlQkXrW9ntTh1wu7IhD5QA==')]) INFO:httpx:HTTP Request: PUT https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?chargeFromGrid=false&serialNo=XXXXXXXX "HTTP/1.1 400 Bad Request" DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'PUT']> DEBUG:httpcore.http11:receive_response_body.complete DEBUG:httpcore.http11:response_closed.started DEBUG:httpcore.http11:response_closed.complete DEBUG:pymyenergi.connection:PUT status 400 DEBUG:httpcore.connection:close.started DEBUG:httpcore.connection:close.complete Traceback (most recent call last): File "/home/vscode/pymyenergi/cli.py", line 4, in cli() File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 238, in cli loop.run_until_complete(main(args)) File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 100, in main await device.set_charge_from_grid(args.arg[0]) File "/home/vscode/pymyenergi/pymyenergi/libbi.py", line 184, in set_charge_from_grid await self._connection.put( File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 120, in put return await self.send("PUT", url, data, oauth) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 75, in send raise MyenergiException(response.status_code) pymyenergi.exceptions.MyenergiException

[trizmark]

This is really puzzling. Can you run the following python code and see what happens? (substitute _usernamehere, _passwordhere and _libbi_serialnumberhere)

import requests
from pycognito import Cognito

u = Cognito('eu-west-2_E57cCJB20','2fup0dhufn5vurmprjkj599041', username='<username_here>')

u.authenticate(password='<password_here>')

headers = {"Authorization": f"Bearer {u.access_token}"}
print("Checking if libbi is enabled to charge from grid")
print(requests.get("https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?serialNo=<libbi_serialnumber_here>", headers=headers).json())

This should produce an output like this;

Checking if libbi is enabled to charge from grid
{'status': True, 'message': '', 'field': '', 'content': {'24047164': True}}

The status should be True of False based on whether you have charging from grid enabled or not.

[videojedi]

vscode ➜ ~ $ python test.py Checking if libbi is enabled to charge from grid {'status': False, 'message': 'Device not found or user does not have access to it!', 'field': ''}

Ah, OK. Because my account is 'sign in with apple', I registered a new account with email/password and shared my original account with full permissions. I am able to login with the new email/password credentials and control my libbi grid charging from myaccount.myenergi.com......

[trizmark]

So, all OK using pymyenergi as well?

[videojedi]

Sorry, don't understand.

I get the exception listed above from pymyenergi when trying to use the credentials from this 'secondary' account. But, if I log into myaccount.myenergi.com with secondary credentials I am able to see and control all devices.

[trizmark]

Apologies, I didn't understand, but now I do! Let me test this at my end and see if it's workable. I'll set up a secondary account and see if there's a way to make it work.

[trizmark]

Haven't had time to troubleshoot the authentication, but started building already on top of this PR! That'll be exposed to HA very soon!

[trizmark]

Just a note that I am running into issues when trying to use Cognito from the HA component due to sync vs async calls. I am trying to work out what's the best way to deal with this, but feel free to chime in if you have experience in this area.

The error in detail:

2023-10-03 09:32:44.457 WARNING (MainThread) [homeassistant.util.async_] Detected blocking call to putrequest inside the event loop. This is causing stability issues. Please report issue to the custom integration author for myenergi doing blocking calls at custom_components/myenergi/pymyenergi/connection.py, line 39: self.oauth = Cognito(_USER_POOL_ID, _CLIENT_ID, username=self.app_email)
2023-10-03 09:32:44.479 ERROR (MainThread) [custom_components.myenergi] Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/botocore/httpsession.py", line 465, in send
    urllib_response = conn.urlopen(
                      ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 714, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 415, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.11/site-packages/botocore/awsrequest.py", line 96, in request
    rval = super().request(method, url, body, headers, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/lib/python3.11/http/client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1297, in _send_request
    self.putrequest(method, url, **skips)
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 219, in putrequest
    return _HTTPConnection.putrequest(self, method, url, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 164, in protected_loop_func
    check_loop(func, strict=strict)
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 151, in check_loop
    raise RuntimeError(
RuntimeError: Blocking calls must be done in the executor or a separate thread; Use `await hass.async_add_executor_job()`; at custom_components/myenergi/pymyenergi/connection.py, line 39: self.oauth = Cognito(_USER_POOL_ID, _CLIENT_ID, username=self.app_email)

[mattburns]

@trizmark , I'm busy at the mo, but perhaps putting it in the __init__ function was a bad idea. Could you move it into the send function instead? In a lazy-load pattern: If not authed yet then auth etc. ?

[trizmark]

@mattburns It's actually fine, I have found a way around it. I tried to move things away from __init__, but Cognito itself was not written to be async. HA provides a solution for it though, so it's all good. I got the charge from grid exposed as a binary sensor at the moment! Working on it to be a proper switch....

[trizmark]

Ugh, I might need to change stuff around as the OAuth token is only valid for one hour. The implementation in the PR is fine for single calls, but before we can use this in the HA component, I'll need to update things. Actually, the solution seems to be super-simple. Just need to wait for an hour to see if it really works.

[trizmark]

Victory! 🎉

[trizmark]

@videojedi To keep the good news rolling: I finally had time to test your special case and you're right, I got the same error. However, there's a solution! After a bit of network sniffing, I figured out what's needed. Can you try the following test with your secondary account?

import requests
from pycognito import Cognito

libbiSerial = ""
appEmail = ""
appPass = ""

u = Cognito('eu-west-2_E57cCJB20','2fup0dhufn5vurmprjkj599041', username=appEmail)
u.authenticate(password=appPass)

headers = {"Authorization": f"Bearer {u.access_token}"}
# grab the list of locations accessible by the user
locs = requests.get("https://myaccount.myenergi.com/api/Locations", headers=headers).json()
# check if guest location - use the first location by default
invId = ''
if locs["content"][0]["isGuestLocation"] == True:
  invId = locs["content"][0]["invitationData"]["invitationId"]
# check if the libbi can charge from the grid
req = "https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?serialNo=" + libbiSerial
if invId != '':
  req = req + "&invitationId=" + invId
d = requests.get(req, headers=headers).json()
chargeFromGrid = d["content"][libbiSerial]

print(f"libbi charge from grid setting: {chargeFromGrid}")

Tech details: with OAuth API endpoints, we need to make a call to api/Locations. From this response we can see if it's a guest location or not. If it's a guest location, we need to include the invitationId in further API calls.

Once you've confirmed that it works for you as well, I can incorporate these changes into pymyenergi.

[videojedi]

@trizmark Brilliant work!

vscode ➜ ~ $ python test.py libbi charge from grid setting: True

Many thanks.

Richard

[trizmark]

For those who like to live dangerously... This is the latest and greatest of the myenergi HA component (plus pymyenergi library). It includes the charge from grid control plus caters for guest account access. Give it a whirl if you want and let me know if it does/doesn't work. I've been running it for a few days, controlling the overnight charging of my libbi. So far, so good! myenergi-20231006.tgz

[videojedi]

Working for me. Thanks everso!

[trizmark]

@mattburns Do you mind if I PR your fork, so we could get all the libbi charge from grid control related pymyenergy changes into this PR?

[mattburns]

@mattburns Do you mind if I PR your fork, so we could get all the libbi charge from grid control related pymyenergy changes into this PR?

Go for it, do whatever you like, I'm a bit busy at the moment to work on this. Thanks

[trizmark]

Go for it, do whatever you like, I'm a bit busy at the moment to work on this. Thanks

OK, PR in. If you merge it, that should update this PR as well, which should set everything up nicely for the HA component update.

[videojedi]

@trizmark Either I've broken it, or sadly the secondary login stuff seems to have stopped working? Even the test code above fails with the following now.

vscode ➜ ~ $ python test.py Traceback (most recent call last): File "/home/vscode/.local/lib/python3.11/site-packages/requests/models.py", line 971, in json return complexjson.loads(self.text, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/init.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/vscode/test.py", line 13, in locs = requests.get("https://myaccount.myenergi.com/api/Locations", headers=headers).json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/vscode/.local/lib/python3.11/site-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

It has been working all week.... something changed on the myenergi server is only possible explanation?

[trizmark]

@videojedi it's not just you - it's myenergi

I use the https://myaccount.myenergi.com/api/Locations API endpoint to determine if I need to add the invitation_id to the OAuth calls. This used to return a JSON object with all the accessible locations. Now it returns a 404. 😠

I'll do some network sniffing in the morning to figure out what I need to change.

[trizmark]

The fix was simple. API endpoint has changed from api/Locations to api/Location (became singular for some reason). Updated HA component, which includes the updated pymyenergi library is attached. I have also issued a PR for @mattburns , so we can get this PR up-to-date. (Hopefully this won't become a regular thing.... 🤞 )

myenergi-20231013.tgz

[videojedi]

Yep all good and confirmed working again. Many thanks

[trizmark]

@CJNE No worries. Right now OAuth is only required if you have a libbi. In the future this will change as I saw a note from myenergi that they're slowly trying to migrate from digest to oauth. Let me make the requested changes.

[trizmark]

@CJNE I've implemented the requested changes. The app_email and app_password parameter is now optional and if they're missing all OAuth stuff is skipped. As part of the libbi only relies on digest, I have left those parts in even if OAuth is missing. So without OAuth you'll get libbi stats, but you won't be able to see the new things, like the 'charge from grid' and 'charge target'. Let me know if this is OK or if I should change anything else.