Smart Meter Texas SSL Certificate Verification Error

[NateEaton]

The problem

Smart Meter Texas integration has been working (including with Energy Dashboard since 40348 was resolved but as of late yesterday, is now reporting Failed to Connect with an SSL Certificate Error.

What is version of Home Assistant Core has the issue?

core-2021.10.2 and core-2021.10.4

What was the last working version of Home Assistant Core?

core-2021.10.2

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Smart Meter Texas

Link to integration documentation on our website

https://www.home-assistant.io/integrations/smart_meter_texas/

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Logger: homeassistant.config_entries
Source: components/smart_meter_texas/__init__.py:49
First occurred: 12:59:56 PM (1 occurrences)
Last logged: 12:59:56 PM

Error setting up entry NateEaton for smart_meter_texas
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/aiohttp/connector.py", line 969, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)  # type: ignore  # noqa
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1111, in _create_connection_transport
    await waiter
  File "/usr/local/lib/python3.9/asyncio/sslproto.py", line 528, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "/usr/local/lib/python3.9/asyncio/sslproto.py", line 188, in feed_ssldata
    self._sslobj.do_handshake()
  File "/usr/local/lib/python3.9/ssl.py", line 944, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'smartmetertexas.com'. (_ssl.c:1129)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 304, in async_setup
    result = await component.async_setup_entry(hass, self)  # type: ignore
  File "/usr/src/homeassistant/homeassistant/components/smart_meter_texas/__init__.py", line 49, in async_setup_entry
    await smart_meter_texas_data.client.authenticate()
  File "/usr/local/lib/python3.9/site-packages/smart_meter_texas/__init__.py", line 196, in authenticate
    await self._init_websession()
  File "/usr/local/lib/python3.9/site-packages/smart_meter_texas/__init__.py", line 147, in _init_websession
    await self.websession.get(
  File "/usr/local/lib/python3.9/site-packages/aiohttp/client.py", line 520, in _request
    conn = await self._connector.connect(
  File "/usr/local/lib/python3.9/site-packages/aiohttp/connector.py", line 535, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/connector.py", line 892, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/connector.py", line 1051, in _create_direct_connection
    raise last_exc
  File "/usr/local/lib/python3.9/site-packages/aiohttp/connector.py", line 1020, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "/usr/local/lib/python3.9/site-packages/aiohttp/connector.py", line 971, in _wrap_create_connection
    raise ClientConnectorCertificateError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorCertificateError: Cannot connect to host smartmetertexas.com:443 ssl:True [SSLCertVerificationError: (1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'smartmetertexas.com'. (_ssl.c:1129)")]

Additional information

This integration has been working fine since July and stopped working last night (2021.10.11 about 9pm CT), still not working this afternoon even after updating Core from 2021.10.2 to 2021.10.4. I am able to log into my account on smartmetertexas.com.

July

Yesterday

[Bostwickenator]

Heh got it. Once you get it to log you in you can grab the token with return JSON.parse(window.localStorage.smt_user)['token']; and use that for the other requests.

[johnavich]

I know this will sound naive but has anyone talked to SMT about this?

I could understand why they would have security concerns from automated access to the superuser dashboards retail electric providers, ercot, and others have. Maybe they could be convinced to lower the controls for retail users.

Additionally I notice this integration is getting the live meter reads. That will cause actual load on their reporting infra. I've been using this data through my own integration where I just pull the last day's 15m interval data once each night. This doesn't cause their system to have to request additional reads.

Maybe they'd be willing to allow access (or drop requirements like a static IP) if the scope of access was lower.

I was on the phone with them last year, got in touch with several directors and stake holders with the various power delivery companies, and they said in no uncertain terms that they will not be lowering any security, or making exceptions to their CDN practices. The ONLY path forward is to either get certificates and have them whitelist your IP, or go with one of the hardware solutions. I plan on doing both, for the first few months or so, to see which is better for me (and to see if the data from one source is reliable enough to ditch the other).

[bengalaviz]

It sounds like they don't want residential customers using their system. There are a lot of requirements preventing people without DevOps experience from setting this up.

My interactions with the India IBM team in 2020, they really didn't help. I was able to create my own root certificate. Then client certificate from that and sent a pfx cert to them. Gave them my public home IP and my personal cloud server static IP (I had plans to use that as a proxy if my home IP ever changed). I just checked the API call and was able to get information back still. My certificate does not expire until 2040. I guess I could write up instructions on how to set everything up. It's been 2 years so I need to try and remember how to create the certificates again.

[twmcelroy]

I'd be very curious on your steps. Ridiculous that they don't want residential customers using their system. But until I can find a good hardware replacement, this is the best thing to track costs. Electricity prices going up and my provider gives me rewards for reducing usage, this will be awesome to track. Or my obsessive need to track everything. I'm starting to lean towards just getting the certificate. I'm sure I could find other uses for it. You API call, are you using node-red or something similar from mrand's repo?

[bengalaviz]

Ok. let me go back and see if I documented how I made my own pfx certificate that didn't cost anything. I was using Insomia Rest client to test their UAT site. But never got around to building something to query their production endpoint. Found this integration and stopped writing my own thing.

I'm a java dev by trade so was going to code something. It could be possible with cURL.. The data from their endpoint "dailyreads" looks like this....

{ "trans_id": "1", "esiid": "XXXXXXXXXXXXXXXX", "registeredReads": [ { "readDate": "01\/01\/2022", "revisionDate": "01\/02\/2022 08:45:11", "startReading": "94246.008", "endReading": "94294.08", "energyDataKwh": "48.033" }, { "readDate": "01\/02\/2022", "revisionDate": "01\/03\/2022 11:19:11", "startReading": "94294.08", "endReading": "94320.433", "energyDataKwh": "26.318" }, { "readDate": "01\/03\/2022", "revisionDate": "01\/04\/2022 09:04:37", "startReading": "94320.433", "endReading": "94344.626", "energyDataKwh": "24.153" }, { "readDate": "01\/04\/2022", "revisionDate": "01\/05\/2022 08:43:44", "startReading": "94344.626", "endReading": "94378.338", "energyDataKwh": "33.675" }, { "readDate": "01\/05\/2022", "revisionDate": "01\/06\/2022 09:00:18", "startReading": "94378.338", "endReading": "94415.368", "energyDataKwh": "36.996" }, { "readDate": "01\/06\/2022", "revisionDate": "01\/07\/2022 08:54:38", "startReading": "94415.368", "endReading": "94459.27", "energyDataKwh": "43.862" }, { "readDate": "01\/07\/2022", "revisionDate": "01\/08\/2022 08:48:06", "startReading": "94459.27", "endReading": "94493.41", "energyDataKwh": "34.105" } ] }

I'll have to poke around with this again and see what data the other endpoints return. It probably won't be until after this weekend that I could update.

[kellcomnet2]

Ok, before I spend too much time building this out, is there any interest in me offering a rest endpoint to get a token by passing in creds which will then 1) spin up a webbrowser instance with a unique working directory 2) pick a random nordvpn server for a proxy so that my ip doesn’t get blocked 3) login to smartmetertexas 4) return auth token for use in further api calls on your end

I would obviously post all code on a public repo

there is the obvious security concern on sending creds to my system.

[Bostwickenator]

Why would you need to use a central system? This could be deployed. That avoids the need for VPNs too.

On Fri, Nov 18, 2022, 10:26 PM Christopher Kelly @.***> wrote:

Ok, before I spend too much time building this out, is there any interest in me offering a rest endpoint to get a token by passing in creds which will then

1. spin up a webbrowser instance with a unique working directory
2. pick a random nordvpn server for a proxy so that my ip doesn’t get blocked
3. login to smartmetertexas
4. return auth token for use in further api calls on your end

I would obviously post all code on a public repo

there is the obvious security concern on sending creds to my system.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/57582#issuecomment-1320794489, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMOHDBMQTBIG7FPJPLOBPDWJBJIZANCNFSM5F3UNMPA . You are receiving this because you commented.Message ID: @.***>

[kellcomnet2]

Mainly because I have no cloud hosting setup to do this, and as eo.webbrowser is a windows only product dc hosting can get expensive

[kellcomnet2]

That being said all code would be posted so people could self host if they want

[kellcomnet2]

Well dang it, I sat down to start coding this morning and the whole site is down again.

[Bostwickenator]

https://gist.github.com/Bostwickenator/2d2a85727ffda0c0e04fa2494fc1daa4 This accomplishes what is needed using Python and Selenium. Selenium and Chrome are available for Linux so I suggest this is a better approach if you want to adapt it.

[miwagner1]

https://github.com/dgtlmoon/changedetection.io/blob/master/docker-compose.yml They have something similar.

[adamfeldman]

Well dang it, I sat down to start coding this morning and the whole site is down again.

SMT seems to do their maintenance/releases on Saturdays. As of noon yesterday the HA SMT integration is working again for me.

[twmcelroy]

Interesting that it was a "maintenance" issue. It is back up and running for me.

[github-infohost-org]

SMT IS BACK WORKING AGAIN IN HA!!! Wonder for how long before they realize someone screwed up and "mistakenly" re-enabled access...

[bengalaviz]

I'd be very curious on your steps. Ridiculous that they don't want residential customers using their system. But until I can find a good hardware replacement, this is the best thing to track costs. Electricity prices going up and my provider gives me rewards for reducing usage, this will be awesome to track. Or my obsessive need to track everything. I'm starting to lean towards just getting the certificate. I'm sure I could find other uses for it. You API call, are you using node-red or something similar from mrand's repo?

Here are my instructions to make a valid client certificate using Windows.

To generate the root cert (In Power Shell)

$cert = New-SelfSignedCertificate -Type Custom -KeySpec Signature -Subject "CN=SmartMeterRootCert"-keyExportPolicy Exportable -HashAlgorithm sha256 -KeyLength 2048 -CertStoreLocation "Cert:\CurrentUser\My" -KeyUsageProperty Sign -KeyUsage CertSign -NotAfter (Get-Date).AddYears(20)

To generate the API client certificate using the root cert above

New-SelfSignedCertificate -Type Custom -DnsName "SmartMeterAPI" -KeySpec Signature -Subject "CN=SmartMeterAPI" -KeyExportPolicy Exportable -HashAlgorithm sha256 -KeyLength 2048 -CertStoreLocation "Cert:\CurrentUser\My" -Signer $cert -TextExtension @("2.5.29.37={text}1.3.6.1.5.5.7.3.2") -NotAfter (Get-Date).AddYears(20)

Open certmgr and navigate to "Certificates - Current User" -> "Personal" -> "Certificates". There you will find the "SmartMeterAPI" Client Authentication certificate. Double click to open the certificate and select the "Details" tab. Click the "Copy to File..." button. Click the "Next" button. Choose "No, do not export the private key". Click "Next". Select "Base-64 encoded X.509 (.CER)". Click "Next". Click "Browse..." and choose where you want to save the exported certificate. Click "Next". Click "Finish".

Now you can share this certificate with Smart Meter Texas.

[twmcelroy]

Thank you. I'm going to save this for the next time they decide to make the integration unavailable.

[dbrigner]

It sounds like they don't want residential customers using their system. There are a lot of requirements preventing people without DevOps experience from setting this up.

My interactions with the India IBM team in 2020, they really didn't help. I was able to create my own root certificate. Then client certificate from that and sent a pfx cert to them. Gave them my public home IP and my personal cloud server static IP (I had plans to use that as a proxy if my home IP ever changed). I just checked the API call and was able to get information back still. My certificate does not expire until 2040. I guess I could write up instructions on how to set everything up. It's been 2 years so I need to try and remember how to create the certificates again.

So, mid-November I got ahead of myself and jumped into all this with SMT and requested access to their official API. I was passed off to the India IBM team who has been very difficult to work with, as you mention. I already had a static IP, so I got an inexpensive cert from Comodo and provided everything they requested. Finally, I was given go-ahead to test, along with their 2.0 documentation, and some very poorly written (and outdated) instructions for configuring and using Postman. With the holidays and several email communications back and forth trying to clarify their email communications (!), I finally sat down to try and figure out how to connect.

I finally got my certs loaded into Postman and after several attempts, got the payload figured out and am now able to get JSON usage data back from SMT.

So the question is "now what?" From what I understand, there is no way to authenticate REST requests within HA using the SSL certs I created for SMT. So has all this effort been moot?

As a non-developer, this has been pretty daunting thus far, but I'm hoping to be able to somehow take advantage of my "official" usage data as a more reliable source than what I've experienced with the HA SMT integration for the unofficial API.

[mrand]

So the question is "now what?" From what I understand, there is no way to authenticate REST requests within HA using the SSL certs I created for SMT. So has all this effort been moot?

Sounds like you may be looking for an "official" access method. There may be other examples out there (I haven't gone looking in a while), but as of 1H2022, my semi-lame repo was reasonably up-to-date: https://github.com/mrand/smart_meter_texas You'd want to avoid any that are labelled "unofficial."

If you are using my curl-based approach in "official_api/home-assistant" subdirectory, you'd presumably want on_demand (I also played with and documented _daily. Not sure there is value in using both).

[NateEaton]

I guess I got a bit complacent after SMT started working again back in November but now it's been down for the past five days. How about everyone else. Also down?

Has anyone come up with a more sustainable way of keeping the integration up and running?

As it happens, I have been working on developing a set of sensors using ESP32 and magnetometer to read my gas meter and water meter. It looks like the same architecture could be used with an optical sensor to read the electricity meter myself instead of relying on SMT. That's looking pretty attractive.

[twmcelroy]

Ya it was going so well for so long. My last day of data was February 10 (minus 11pm). I know sometimes it drops for an hour or 2 but clumps that into the next hour. Not a big deal. But February 10 11 PM has no data, a large spike February 11 at 8 am, normal amount at 9am and then nothing.

I'd also be very interested in the gas and water meter ESP32. I have worked with it yet but I'd love to bring that into my home. Well that and figuring out how to do water cutoff in texas when your cutoff valve is out in the yard.

I guess the other part is to see if the Node-Red implementation is still working. I deactivated that one previously.

[thechriswick]

Been down for past 5 days as well. I use the Node Red configuration. @NateEaton I would be interested in learning more about your progress with the ESP32 setup.

[logandirks]

I believe this is related to this thread but I'm encountering the following:

Logger: smart_meter_texas Source: /usr/local/lib/python3.10/site-packages/smart_meter_texas/init.py:312 First occurred: 9:17:59 PM (1 occurrences) Last logged: 9:17:59 PM

Failed to lookup CA Issuers URI value

Any help would be outstanding!

[NateEaton]

Been down for past 5 days as well. I use the Node Red configuration. @NateEaton I would be interested in learning more about your progress with the ESP32 setup.

@thechriswick, it's still a work in progress, a learning experience. Programming hadn't been part of my job description for quite a while (retired as a technology manager) but I've always enjoyed the opportunity to apply that skill set on home projects and I'm enjoying learning as I go. Rather than get into details and go off-topic here, I went ahead and added my goals to this repo: EspUtilityMeter.

[spraguej248]

It is operational again. Just had to restart the addon. I think they decide on Saturday whether to allow connection for the week? :)

On Fri, Feb 17, 2023 at 10:24 PM Nate Eaton @.***> wrote:

Been down for past 5 days as well. I use the Node Red configuration. @NateEaton https://github.com/NateEaton I would be interested in learning more about your progress with the ESP32 setup.

@thechriswick https://github.com/thechriswick, it's still a work in progress, a learning experience. Programming hadn't been part of my job description for quite a while (retired as a technology manager) but I've always enjoyed the opportunity to apply that skill set on home projects and I'm enjoying learning as I go. Rather than get into details and go off-topic here, I went ahead and added my goals to this repo: EspUtilityMeter https://github.com/NateEaton/EspUtilityMeter.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/57582#issuecomment-1435479848, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF5XVHUCYFUUDUMTYGGZCL3WYBFILANCNFSM5F3UNMPA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[dbrigner]

So the question is "now what?" From what I understand, there is no way to authenticate REST requests within HA using the SSL certs I created for SMT. So has all this effort been moot?

That's sort of where I am, too. According to SMT India, residential users "can get the json format of usage for a single ESIID meter without delivery mode from an API client tool (Postman/SoapUI). It is synchronous request." I was able to use PostMan to successfully add the certs, modify the payload, and can get valid data back. But where to go from there..? Doesn't sound like this is possible in HA.

[Lanyx01]

For those curious, it appears to be working again. Started working yesterday for me.

[johnavich]

@dbrigner That's all for the Official API, which i switched over to several months ago. The official API did not go down during this outage. If your interested in this method, i'll post something to my personal git and share it here. In essence, i get the raw kwh from SMT via node-red, and post it using MQTT. This has been very reliaable, as long as i remember to send them my SSL Certificate every ~60-90 days. its not relavent to this particular issue, but i don't think this issue will ever by solved via code, as its SMT's website that is causing issues.

[NateEaton]

@dbrigner That's all for the Official API, which i switched over to several months ago. The official API did not go down during this outage. If your interested in this method, i'll post something to my personal git and share it here. In essence, i get the raw kwh from SMT via node-red, and post it using MQTT. This has been very reliaable, as long as i remember to send them my SSL Certificate every ~60-90 days. its not relavent to this particular issue, but i don't think this issue will ever by solved via code, as its SMT's website that is causing issues.

I would be interested in the details. To be able to have direct access to the official API, whether using Node-Red or anything else with REST & MQTT capabilities, and no longer be at the mercy of their website seems attractive.

[thechriswick]

@dbrigner That's all for the Official API, which i switched over to several months ago. The official API did not go down during this outage. If your interested in this method, i'll post something to my personal git and share it here. In essence, i get the raw kwh from SMT via node-red, and post it using MQTT. This has been very reliaable, as long as i remember to send them my SSL Certificate every ~60-90 days. its not relavent to this particular issue, but i don't think this issue will ever by solved via code, as its SMT's website that is causing issues.

I would very much be interested in the API git.

[dbrigner]

@dbrigner If your interested in this method, i'll post something to my personal git and share it here.

@NateEaton Yes, I’m very much interested. Thank you!

[NateEaton]

@dbrigner If your interested in this method, i'll post something to my personal git and share it here.

@NateEaton Yes, I’m very much interested. Thank you!

@dbrigner, apologies... I was replying to @johnavich but since he had addressed you I guess it looked like I was addressing you.

@johnavich could you please share the details you mentioned?

[dbrigner]

@dbrigner, apologies... I was replying to @johnavich but since he had addressed you I guess it looked like I was addressing you.

Yes, I see that now. Thanks for clearing that up. I should know better than to try and pull quotes and post from my phone, without my glasses...

At any rate, @johnavich I would be very interested in your solution.

[johnavich]

https://github.com/johnavich/SMT-NodeRed-Official

I attempted to add notes to the readme, but i am not a document writer. If you have questions regarding the config, please create an issue on that Repo, and we can leave this one alone. Personally, i think we should let this thread die, as we're not going to be able to get SMT to cooperate with us.

[NateEaton]

Thanks, @johnavich.

I agree... and as I opened this issue, I'll go ahead and close it since the issue I opened it for has been solved as good as it can be. Thanks to all who contributed.