search

unlobito / ha-hildebrandglow

PRE-RELEASE. HomeAssistant integration for the Hildebrand Glow smart meter HAN for UK SMETS meters.
MIT License
54 stars 32 forks source link

Non-Glow Stick MQTT devices aren't detected #31

Closed stav242 closed 3 years ago

stav242 commented 3 years ago

installed through HACS (restarted) and added via Integration.

used my account username and password. the setup page shows..

Success! Created configuration for .

and the Integration shows its failed to start... logs show...

Logger: homeassistant.config_entries Source: custom_components/hildebrandglow/glow.py:93 Integration: Hildebrand Glow (documentation) First occurred: 9:37:38 (1 occurrences) Last logged: 9:37:38

Error setting up entry for hildebrandglow Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 293, in async_setup result = await component.async_setup_entry(hass, self) # type: ignore File "/config/custom_components/hildebrandglow/init.py", line 30, in async_setup_entry await hass.async_add_executor_job(glow.retrieve_cad_hardwareId) File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/config/custom_components/hildebrandglow/glow.py", line 93, in retrieve_cad_hardwareId self.hardwareId = cad["hardwareId"] KeyError: 'hardwareId'

unlobito commented 3 years ago

Hi @stav242, like #30, this appears to still be a duplicate of #26.

If you're not using DCC Other User to give Hildebrand access to your smart meter(s), could you reopen this issue with details of the Consumer Access Device being used?

danstreeter commented 3 years ago

I am also now getting this same error messages. Using the followign meter on the same network as the homeassistant installation, credentials entered on integrtion configuration confirmed as working via both postman to their API endpoint and also via MQTT explorer to my MQTT feed.

My CAD meter: https://shop.glowmarkt.com/products/display-and-cad-combined-for-smart-meter-customers

Cannot seem to re-open this issue, must be permissiong but @unlobito, as above. Thanks.

stav242 commented 3 years ago

i had a non HACs custom repository for hildebrand working ok, then through it should move over to the HAS one - cant remember what the old one was but it did work ok.

It seems this code version doesnt recognise the CAD device.

i do have a separate node-red flow which is getting my electric consumption data from Hildebrand - its my fall back now this has stopped working. can i provide any info to help get this code updated to with with the CAD?

On Tue, 15 Jun 2021 at 12:31, Dan Streeter @.***> wrote:

I am also now getting this same error messages. Using the followign meter on the same network as the homeassistant installation, credentials entered on integrtion configuration confirmed as working via both postman to their API endpoint and also via MQTT explorer to my MQTT feed.

My CAD meter: https://shop.glowmarkt.com/products/display-and-cad-combined-for-smart-meter-customers

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/unlobito/ha-hildebrandglow/issues/31#issuecomment-861420917, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOJGGV76ZZL6ULDIF454TDTS422XANCNFSM46WZHFBQ .

danstreeter commented 3 years ago

@unlobito - Happy to help debug this issue, will be working and on my laptop most of the evening (UK time) so can provide immediate debug if you let me know what you need, and how I can do it.

Currently running Home Assistant 2021.6.0b5 on Docker

unlobito commented 3 years ago

Hey @danstreeter @stav242, thanks for providing more details about what's gone wrong here.

The current (https://github.com/unlobito/ha-hildebrandglow/releases/tag/0.1.1-beta) release is hardcoded to search for the Glow Stick at https://github.com/unlobito/ha-hildebrandglow/blob/mqtt/custom_components/hildebrandglow/glow.py#L89-L91 by its deviceTypeId (1027b6e8-9bfd-4dcb-8068-c73f6413cfaf). Extending this to also list the other CADs should be enough, but I'm not sure what their IDs are.

Ideally, the log message here would indicate which deviceTypeIds have been found (you could use debugpy to attach a debugger and see which values it encountered), but calling the https://api.glowmarkt.com/api/v0-1/device endpoint while authenticated would also indicate the right value.

danstreeter commented 3 years ago

Just ran the API request as that was much quicker and I get 5 instances of a deviceTypeId within the response payload.

Happy to share the payload here if it doesnt have any personal / identifiable / personal references in it?

danstreeter commented 3 years ago

With regards to the debugpy option: I'll look at implementing that which may aide now and in the future however can you confirm my thinking that i'd need the code running within the running homeassistant container locally to provide breakpoint and debug functionality?

unlobito commented 3 years ago

Hey @danstreeter,

Just ran the API request as that was much quicker and I get 5 instances of a deviceTypeId within the response payload.

Happy to share the payload here if it doesnt have any personal / identifiable / personal references in it?

Brilliant! Some of these will be for the smart meters themselves (and can include your MPAN, so would lean against fully posting it to the internet). I'm not sure what the non-Glow Stick CADs will be labelled as, but here's a redacted version of my response so you can find the CAD within your response. I'd expect your CAD to have description set to ~roughly "Zigbee Glow In Home Display" based on the product page.

{
  {
    "protocol": {
      "protocol": "Zigbee SEP1.1b",
      "sensors": []
    },
    "parentHardwareId": [],
    "tags": [
      /* ... */
    ],
    "hardwareIdNames": [
      "MAC",
      "serialNumber",
      "EUI"
    ],
    "active": true,
    "deviceTypeId": "1027b6e8-9bfd-4dcb-8068-c73f6413cfaf",
    "ownerId": /* ... */,
    "hardwareIds": {
      /* ... */
    },
    "hardwareId": /* ... */,
    "description": "Zigbee Glow Stick",
    "configuration": {
      "openHan": {
        "adaptorType": "smart"
      }
    },
    "deviceId": /* ... */,
    "updatedAt": "2019-08-31T09:42:13.570Z",
    "createdAt": "2019-08-31T09:42:13.570Z"
  },
  /* ... */
}

With regards to the debugpy option: I'll look at implementing that which may aide now and in the future however can you confirm my thinking that i'd need the code running within the running homeassistant container locally to provide breakpoint and debug functionality?

Yeah, since the retrieve_devices method relies on having an active session token to make the HTTP request. You could also call glow.py directly to the same effect.

from .glow import Glow

glow = Glow(APP_ID, data["username"], data["password"])
glow.authenticate()

from pprint import pprint
pprint(glow.retrieve_devices())
danstreeter commented 3 years ago

Aha your reply literally came in as I was typing this...

Ok, not sure if this helps any more - but reading around the glow.py file you forwarded above, and determining that you lock into the {hardwareId} for MQTT topic - and then looking at my MQTT Explorer subscription settings which references the topic to subscribe to, taking the id that you reference as {hardwareId} on line 22, searching for that within the returned payload, I end up with an object is only referenced once...

Please find my redacted single object as follows (there are many more objects like this in the payload however, none with reference to my MQTT topic/hardwareId:

{
    "protocol": {
        "protocol": "Zigbee SEP1.1b",
        "sensors": []
    },
    "parentHardwareId": [],
    "tags": [
        /** ... */
    ],
    "hardwareIdNames": [
        "MAC",
        "serialNumber",
        "EUI",
        "ic",
        "deviceModel"
    ],
    "active": true,
    "ownerId": "...GUID...",
    "hardwareIds": {
        /** ... */
    },
    "deviceTypeId": "b91cf82f-aafe-47f4-930a-b2ed1c7b2691",
    "hardwareId": "...ALPHA_STRING...", // This is what my MQTT Explorer topic references that you call {hardwareId} on line 22 of glow.py
    "description": "Glow Display SMETS 2",
    "deviceId": "...GUID...",
    "updatedAt": "2021-02-24T10:35:48.470Z",
    "createdAt": "2021-02-24T10:35:48.470Z"
},
unlobito commented 3 years ago

Hey @danstreeter, could you try 2e7c3bdd644b9dc7260254547e158d72f031e43e and see if that's able to identify your CAD?

danstreeter commented 3 years ago

Sure, how do I force a release from that commit?

Go in manually and checkout the code to the custom_components directory? (not using the HACS release process)

unlobito commented 3 years ago

Yep 👍 I've also released it as https://github.com/unlobito/ha-hildebrandglow/releases/tag/0.1.2-beta if that's easier.

danstreeter commented 3 years ago

OK went real raw and updated that file with the changes live on the server and restarted home assistant, now connects and gets four entities... Checking for entity data now.

danstreeter commented 3 years ago

image

image

When adding all four entities to an entity card: image

danstreeter commented 3 years ago

Reverted my manual change, updaed to 0.1.2-beta and restarted, now get a much more reasonable consumption reading... 422 Watts not - near on 10 nuclear explosions! lol

Still nothing for the other three.

unlobito commented 3 years ago

Great! I think there are some separate issues here for the Unknown values-

I think the initial electricity read was caught out by the integration not properly handling the formatting data returned over MQTT, instead just directly returning the received value.

unlobito commented 3 years ago

@stav242 Could you give https://github.com/unlobito/ha-hildebrandglow/releases/tag/0.1.2-beta a try and see if that's also able to pick up your CAD? Thanks!

danstreeter commented 3 years ago

Awesome, thanks for the rapid fix.

I've been using custom sensors setup via a MQTT bridge as outlined here: https://gist.github.com/ndfred/5bb51b567f8cfaf2bb06da6393169321 until now - but nice ot have an integration over some 'rather long' value template code!

I'm on 0.1.2-beta already - dropped onto it soon after you mentioned you'd pushed it.

stav242 commented 3 years ago

hi, no luck.

so it logged in ok and found my CAD this time, but not getting any data.

what can i provide to assist you?

On Tue, 15 Jun 2021, 18:57 Harley Watson, @.***> wrote:

@stav242 https://github.com/stav242 Could you give https://github.com/unlobito/ha-hildebrandglow/releases/tag/0.1.2-beta a try and see if that's also able to pick up your CAD? Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/unlobito/ha-hildebrandglow/issues/31#issuecomment-861714738, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOJGGRPU532QMLTECOZUKLTS6IARANCNFSM46WZHFBQ .

stav242 commented 3 years ago

does this help ?- output from my node-red flow which is currently working...

6/15/2021, 7:30:26 PMnode: 6805a9a7.444998 https://sp8.duckdns.org:8123/api/hassio_ingress/p7eVIh_Im3Qq39j18kubZvi8usKyYhs37O7sJgZLYes/#msg.payload : array[1] array[1] 0: object clone: false active: true veTypeId: "6b61c150-34f8-4e1e-9b65-1df2a63611a6" resources: array[2] 0: object resourceId: "284b944c-4e85-40ba-a3da-adea588695ed" resourceTypeId: "ea02304a-2820-4ea0-8399-f1d1b430c3a0" name: "Smart Meter, electricity consumption" 1: object resourceId: "08d98553-ddf7-4af1-9f39-ec88bf85c134" resourceTypeId: "b4158501-a678-484a-837a-874194d3bd48" name: "Smart Meter, electricity cost" ownerId: "c5e9a81e-f822-4b3d-ae5c-fade67449ac8" applicationId: "b0f1b774-a586-4f72-9edd-27ead8aa7a8d" name: "Smart Home Virtual Entity" veChildren: array[0] veId: "6662cc82-a386-40fc-8505-52d07cec8937" updatedAt: "2020-07-03T10:39:38.130Z" createdAt: "2020-07-03T10:39:12.206Z"

6/15/2021, 7:30:26 PMnode: electric_consumption https://sp8.duckdns.org:8123/api/hassio_ingress/p7eVIh_Im3Qq39j18kubZvi8usKyYhs37O7sJgZLYes/#msg : object type: "nodered/entity" server_id: "d56067c4.d66728" node_id: "8146a288.bbe9a" state: 2625 attributes: object kw: 0 id: 109

On Tue, 15 Jun 2021 at 18:57, Harley Watson @.***> wrote:

@stav242 https://github.com/stav242 Could you give https://github.com/unlobito/ha-hildebrandglow/releases/tag/0.1.2-beta a try and see if that's also able to pick up your CAD? Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/unlobito/ha-hildebrandglow/issues/31#issuecomment-861714738, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOJGGRPU532QMLTECOZUKLTS6IARANCNFSM46WZHFBQ .

unlobito commented 3 years ago

Hey @stav242, thanks for the info- do you have MQTT working in Node-RED? You might need to ask Hildebrand to enable this on your account. The flow you've posted appears to be coming from the REST API.

stav242 commented 3 years ago

i see, i dont have MQTT setup specifically on the CAD, didnt know that was needed. ok ill ping off a message to hildebrand and get them to enable it. and post back when i find out more . thanks.

On Tue, 15 Jun 2021 at 19:37, Harley Watson @.***> wrote:

Hey @stav242 https://github.com/stav242, thanks for the info- do you have MQTT working in Node-RED? You might need to ask Hildebrand to enable this on your account. The flow you've posted appears to be coming from the REST API.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/unlobito/ha-hildebrandglow/issues/31#issuecomment-861741265, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOJGGQ22UZOOZKWWUMYIDTTS6MYFANCNFSM46WZHFBQ .

stav242 commented 3 years ago

Ok. MQTT is enabled now on the CAD. I'm getting data but looks like it needs adjusting..

Top2 cards are MQTT, bottom one is real data via rest API

On Tue, 15 Jun 2021, 19:37 Harley Watson, @.***> wrote:

Hey @stav242 https://github.com/stav242, thanks for the info- do you have MQTT working in Node-RED? You might need to ask Hildebrand to enable this on your account. The flow you've posted appears to be coming from the REST API.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/unlobito/ha-hildebrandglow/issues/31#issuecomment-861741265, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOJGGQ22UZOOZKWWUMYIDTTS6MYFANCNFSM46WZHFBQ .