update tracking issue

[richo]

Hey Folks,

Sorry for the ping this just seemed like the easiest way to keep you updated. FranklinWH have officially released their update. The good news is it hasn't broken anything I have already released, and the better news is that assuming everything went well, it should create a path toward rolling out some more features (realtime stats, smart switch control, mode control).

I will be home next week, and plan to work on this sometime around then. I'll continue updating this issue with progress and might ask if some of you are willing to test some things. Also, please let me know if there's an easier way to keep folks updated. I really appreciate you using this (I literally wrote it for no other reason than I needed it) but don't want to hassle people.

Thanks!

/cc @slackwilson @TheSwert @npdsomerhayes @bradyr

[bradyr]

interesting timing of this post -- i had to re-issue a API token on my HASS integration for my aGate this morning. i woke up and noticed i wasnt getting any telemetry from the agate. restarted home assistant for giggles and that didnt fix. re-issued myself a token using that trusty ole' script and that fixed it upon a 2nd restart of home assistant.

[richo]

Fascinating. I just got the email from Franklin, and after fighting for a while to get tailscale working on my hass instance I was able to have a look- it certainly appears that mine is still working with my ancient token which is vaguely surprising to me.

Out of curiosity when yours wasn't working, was it showing stale data or no data?

[npdsomerhayes]

Hi everyone

Firstly I am more than happy to do testing.

I also had to regenerate my token this morning, I need to double-check what time it went offline as I also did a HASS update yesterday so it could be related to that.

below is the error shown in my logs this looks the same as when I was originally having trouble with my access token.

`Logger: homeassistant.helpers.entity Source: helpers/entity.py:941 First occurred: 6 June 2024 at 21:30:48 (711 occurrences) Last logged: 09:20:49

Update for sensor.franklinwh_home_load fails Update for sensor.franklinwh_state_of_charge fails Update for sensor.franklinwh_grid_use fails Update for sensor.franklinwh_battery_use fails Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 941, in async_update_ha_state await self.async_device_update() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1291, in async_device_update await hass.async_add_executor_job(self.update) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/franklin_wh/sensor.py", line 114, in update stats = self._cache.fetch() ^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/franklin_wh/sensor.py", line 74, in fetch self._fetch() File "/config/custom_components/franklin_wh/sensor.py", line 67, in _fetch self.data = self.update_func() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/franklinwh/client.py", line 125, in get_stats data = json["result"]["dataArea"]

KeyError: 'result'
`

[bradyr]

id have to double check my logs but im pretty sure i got the same errors that @npdsomerhayes was getting when i lost telemetry with my aGate (at around 4:25AM, PST/Caifornia time). according to my sensors it just "flat lined" and showed stale/previous known good values on the graphs but also those error logs.

semi related, when i lost telemetry , it broke the various template sensors i created based on the franklin integration ones -- convert kW to Watt and i also have some to generate kWh data as well as splitting the battery charge and discharge data so that i can integrate it all into the energy section of home assistant. upon regenerating the api token, and bouncing home assistant, all was good with my custom sensors.

[slackwilson]

@richo that does sound good, and I'm absolutely up for testing. In fact I still have my vmware/virtualised "test" ha install so easy for me to try stuff without breaking my main install. If you have anything you're a bit iffy about I'm happy to try it there 1st, and happy to test anything you need.

In terms of how to contact, this is fine and I'm also fine being emailed direct, you already have my email as does @npdsomerhayes

My token is still working for me so far, have not had to recreate the token, at least not yet anyway.

EDIT: actually, weirdly I just had to recreate the token on my test system even though the same token is working fine on my main system. I just ran up the test system to make sure it was working and ready for any testing coming up and sure enough it wasn't. After recreating the token now it's ok.

[slackwilson]

FYI in case anyone might find it useful, on my test HA system I had a play with turning on the kWh usage for grid in & out, battery in &out and Solar for use in the energy page of home assistant. on my main HA I already have tracking for Grid in & out using an eagle 3 device, and for Solar direct from my goodwe inverter, but for the sake of trying it out thought I would see what it was like getting all data purely from the franklin system.

works fine, on top of getting battery charge/discharge - if you have a solar system that doesn't integrate into HA easily, and more likely if you don't already have a method to track energy to/from the grid into HA this is a way to get there.

I did have to create some helpers because the kWh data resets just after midnight each day which HA does not like (just before is fine but not after)

If anyone wants more detail on how I set it up just let me know, I could document how it did it if it's useful to anyone.

By the way, the data is correct, it was a terrible winters day here yesterday so I generated hardly any solar hence the data looks a bit sad.

[npdsomerhayes]

Not sure if anyone else had this issue but I had to regenerate my token again this morning. It looks like it went off line at around 09:20 New Zealand time (UTC +12). I am not sure if it is related but it is a week since the last time it happened!

[bradyr]

@npdsomerhayes yeah i had to regenerate my token this morning as well. looks like mine went offline at approx 10:10AM Pacific time (UTC +7). i'm going to set myself some kind of reminder for next week and see if it flat lines again. if so, i'm sure we can script it or do some kind of cron job to have it update the token weekly.

[slackwilson]

@npdsomerhayes @bradyr interestingly looks like I have to regenerate a token too.

Not sure what it means, but I am using the same token on my test HA and my main HA, my test HA went offline around 9am AEST (UTC+10) however the same token is still working on my main HA system.. maybe it only checks occasionally and my main system will go offline soon.

[richo]

Hey thanks for your patience and for the updates. Interestingly, my token hasn't expired, but the other automations I had around it broke sometime today. I haven't had a chance to poke much, and unfortunately I had a bit of a family emergency today but I think I'll have something for y'all to test sometime next week.

Thanks

richo

[slackwilson]

quick update on the token, this morning my main HA system needed the token updated - I just copied the same one I recreated 3 days ago for my test HA.

[richo]

ok so a couple of things. I think my token finally got expired. I think franklin are actually moving a little slower than they said with the rollouts.

On the tokens front, I really wanted to just use tokens in the config to avoid having to put your username and password in but clearly that's going to be too flaky. I'll rollout an update sometime this week that lets you just put your credentials in there and it'll issue its own tokens as it goes along to at least solve that piece.

[richo]

Ok finally got a chance to have a look at what they've done. It actually makes things a lot more straightforward from the perspective of this module, although on balance I think I'm just gunna rewrite the whole thing from scratch and bump the version number up a ton instead of trying to carry this forward.

[slackwilson]

@richo thanks for the updates, if I can help in anyway let me know.

FYI - I identified these 2 from your client.py in the franklinwh-python repository and edited Sensor.py to gather the data for use in the inbuilt HA energy page (I showed pics a few messages earlier in this issue discussion)

class Totals: battery_charge: float battery_discharge: float

If you're gonna rewrite I'd be keen in still having access to those two :)

[richo]

Feel free to open a PR with changes like that! There's no real downside to gathering extra data (You're fetching it all from upstream anyway).

I will make sure they survive the rewrite though :)

[slackwilson]

I will do, I figured no point yet since you already mentioned rewriting it anyway, but after you rewrite I'll submit :)

[richo]

@slackwilson gotcha. I guess two things, first of all my plan was to rewrite the underlying python library so you can safely PR stuff in here for the sensor and switch, but also I wound up getting most of it working on top of what I had.

I'm about to push an update now which should get all the sensor stuff working again (and it issues it's own tokens! So you can just put your username and password in the config and never have to worry again!). The mode changing + smart switch stuff might be a little ways out :)

[richo]

Update is up on main! And it depends on the latest version you need of the python module so you shouldn't have any of the cache clearing issues we had last time. I just updated the README with some information about how to update your config but it should only be a case of needing to change it to use username: and password:. My advice would be to put at least the password in secrets.yaml.

I'll try to get switches and mode changing working shortly. You'll likely get mode changing first because it actually looks easier now that I'm looking at it more closely.

[slackwilson]

@richo I've installed the latest on my test system, and so far no issues at all, the username/password instead of token seems to be working great, and the data is coming through fine currently. If I see any issues I'll report but so far all good!

Appreciate your efforts, and anything I can do to help let me know.

[npdsomerhayes]

@richo I have the update installed and all seems to be working well. I will let you know if I discover any issues.

Thanks for all your hard work.

UPDATE: recloned the repository to get @slackwilson sensors. That data is also coming in

[bradyr]

@richo @slackwilson @npdsomerhayes

I also wiped and did a fresh clone of the latest updates -- it's working great using the new username/password method.

the only oddity i'm seeing for my instance is that my smart switch status is showing them all as "off" when they are actually "on" . now, i do not know if i need to toggle the state in order for it to get refreshed. I'll check when i'm actually home and standing in front of the aGate so that i can hear the relays when i attempt to toggle them via HASS. my smart switch configuration is 3 switches: two 20A/120V single pole and one combined 20A/240V double pole for my HVAC (..this is US/American power)

as a side bonus, if anyone is interested in LIVE data for battery charging / discharge (i.e. power device class) for live monitoring, here are 3 helpers i created for this task:

live -- battery charging:

{{ min(states.sensor.franklinwh_battery_use_watts.state | int, 0) | abs | default(0)}}

live -- battery discharging:

{{ max(states.sensor.franklinwh_battery_use_watts.state | int, 0) | default(0)}}

live -- battery use in watts:

{{ states('sensor.franklinwh_battery_use')|float * 1000 }}

for all:

unit = W device class = Power State class = measurement

i convert to WATTS since it lines up nicely with my emporia vue data. you will need to adjust the names of the template sensors to match whatever you come up with.

thanks all for the great work on this integration.

[bradyr]

@richo semi-unrelated, i'm going to let these recent changes settle but since i have a generator also integrated into my aGate and i did see some generator references in that api response dump, i might fork this and do a PR for adding in some generator sensors. even if we very rarely use the generator, having the sensors ready to log data is a cool thing.

thanks again for all the hard work.

[richo]

Sorry I thought I wrote that part out but I might have missed it.

I don't have switches working yet, but I didn't want the entities to disappear so they're kinda dummies for right now. If you care about seeing the state but not altering it I can definitely wire that up today though. And yes, please PR away!

[bradyr]

@richo showing the active state of the smart switches, even if it's read only, would be nice but it doesnt have to be at the top of your list-of-things to-do.

the generator info is also low priority only cuz we're typically not on generator. having the sensors available is cool tho so i might take a stab at doing up the code for it unless someone else beats me to it.

thanks!

[richo]

@bradyr showing the state of smart switches should be up on main now! My setup didn't require config changes so if it worked before, you should be able to just update and have it start working again.

I will likely try to figure out the logistics of putting it up on the store thing next week and get closer to a stable channel/beta channel situation.

[richo]

I created a new issue for generator things at #10 just to help me keep track of everything

[bradyr]

@richo confirming that smart switch state is good to go now on my instance as well. thanks for the quick turn around on that one.

[richo]

I just pushed an update that will make manipulating smart switches work again.

I also had to yank all the old versions of the python module because of a stupid mistake I made. I don't think this will actually affect anything, everything should pull the latest version by default anyway but if you see errors relating to old version of the module (Current is 0.0.7) let me know and I'll try to fix it.