10 sec sensors become unavailable when HA restart

[jnrksv]

After upgrading hass-AMS to 1.04 all 10-sec-package sensors becomes unavailable when (and everytime) HA restarts. The hour-package sensors seems to be OK and updates every hour.

In HA it looks like this:

I have to delete the integration, restart HA and add the integration again to get it up and running.

This is the debug log after a restart with both 10-sec- and hour-package included. As far as I can see, everything looks normal (but I don't know what to look for). ha-debug-log.txt

[turbokongen]

Well that's a tricky one. As you stated, everything looks to be fine. Data is read from port, and is valid. The sensors are registered, and updating. I cannot reproduce this on any of my two servers. If you downgrade to v1.03 and reboot multiple times, does it work like expected then? After downgrading and confirming/not confirming the bug. Do an upgrade again and see if the bug is still there.

[jnrksv]

Downgrade to 1.03 didn't help. Exactly the same behavior. Actually, I thought I had figured it out. After I had deleted the integration, the ams sensors was still present in the device registry (but not in the entity registry). I thought it might be some old junk from an early installation that caused this trouble. I deleted the entries from the .storage/core.device_registry file, restarted HA and did a new installation of 1.04. But that didn't help either. I think I give up on this one and live with it for the time being. Thank you anyway.

[turbokongen]

The entity and device registry was actually my initial thought. Anyway, my remedy for problems in those files have been to stop hass, delete them and then start hass. They do get recreated at hass start.

I close this issue, but if more people find this issue, please reopen it :100:

[turbokongen]

Could you try this branch: https://github.com/turbokongen/hass-AMS/tree/missing_sensors There are some change regarding entities setup and device setup, I believe this might have caused it.

[jnrksv]

Seems to be the same issue with the "missing_sensors branch" too. Happens after first restart. The integration looks like this (I assume icon changed to flash because of "device_class: power"). Attachs some lines from the debug log after the first restart. Reading data from port seems to be okay but sensors still unavailable... missing_sensors_branch_log.txt

[turbokongen]

Could you run the latest code from the same branch. Can you save a log from the setup of the integration, and then save a log from the next restart. Then I can compare the two to see where it goes wrong.

[jnrksv]

Logs as wanted. missing_sensors_branch_debug_log_2.txt

[turbokongen]

Ok, I do see what is happening here, but I do not understand why. The sensor registration does not happen in the "First 10sec readout" log. I might just try some stuff, and ask you to try the code. Like I said, I can't reproduce this on any of my two running machines. I'll post some changes, a little later.

[turbokongen]

Please try the latest pushed code in missing_sensors branch. There is no need to get data for the cummulative sensors.

[jnrksv]

Tried the latest branch. 10sec sensors still unavailable after restart (sorry, didn't notice your "no need to get data for the cummulative sensors" comment before now, it's in the attached log file. missing_sensors_branch_debug_log_3.txt

[turbokongen]

Ok, thanks for the feedback. Are there any of the released versions that do work for you?

[jnrksv]

That's a good question. I actually thought my problems started when upgrading to 1.04. But downgrading to 1.03 didn't solve it, so it has to be something else. Actually I have used all version since your first release and haven't had this issue before version 1.04. That said, at about the same time as the problems started, I upgraded HA to 0.107.7 and ran into a lot of trouble with other custom components using the pendulum package, and my HA got a bit messy for a day or two. Hellowlol (owner) fixed his components and all works okay now. I had a chat with him this afternoon and came to mention this. He asked me to test his "stuff-branch" of your hass-ams. I did and this branch actually works! Now I have restartet HA several times and the 10sec sensors updates immediately after restart. hass_ams_Hellowlol_stuff_branch_debug_log_3.txt I have no idea why his "stuff-branch" works on my system and your latest versions don't...

[turbokongen]

The thing that happens are that the sensor setup never get's executed on yor restarts, but it does when the hourly sensors comes in. The "hub" does not execute the setup step in sensor setup. In other words, the callback does not get called. I will have to dig a bit more to find what is causing this. Or if any of you or @Hellowlol knows what is the cause give me a hint 💯

[turbokongen]

Do you have the integration activated by configuration.yaml ? Or an entry in configuration.yaml of ams?

[Hellowlol]

Mine support both version. He seems to be using config flow.

[jnrksv]

Right. Nothing in configuration.yaml.

[turbokongen]

There is something fishy going on in @jnrksv case, because: https://developers.home-assistant.io/docs/config_entries_index/#setting-up-an-entry async_setup_entry does never gets called. That is the reason for this. So the stuff branch will work, because there is also a setup call in the async_setup method. I'm trying to figure out why.

[turbokongen]

@jnrksv Please try the latest from missing_sensors branch. I made a dirty hack to see if that changes anything.

[jnrksv]

Tried it, but unfortunately 10sec sensors still unavailable after first restart. missing_sensors_branch_debug_log_4.txt

[turbokongen]

Ok, a little update here. Suddenly I didn't get any sensors either. I then examined the .storage directory. And found entries in each of the following files. This should work with the v1.05 also. I deleted the following files to get it working again: (They do get recreated, but backup if critical) core.config core.config_entries core.device_registry core.entity_registry

[Hellowlol]

I’m pretty sure that the bug is somewhere in https://github.com/turbokongen/hass-AMS/pull/15 On mobile atm so I havnt tried debugging anything, but the logic for added sensors was changed in this pr

[turbokongen]

@Hellowlol I would have thought that too, but v1.03 did not contain that specific change. So that change is already ruled out.

[Hellowlol]

@jnrksv what was the latest version that worked?

[jnrksv]

Another good question. I'm pretty sure I didn't noticed this behavior in any of the versions before 1.04. I was pretty sure I've used all version of hass-ams because I always install new updates as they become available in HACS. But that was until you asked. Now I'm not so sure. Keep in mind I'm an old man and I can't always remember what I did yesterday.

[turbokongen]

@jnrksv can you try to delete the four files mentioned earlier. Stop hass and backup if they a critical to you. They will be recreated at startup. core.config core.config_entries core.device_registry core.entity_registry

[jnrksv]

Done (accept I don't have any file named core.config). After first restart 10sec sensors were still available and updating every 10th second. After second restart the 10sec sensors became unavailable again.

[turbokongen]

Added some new code to try. Please test.

[jnrksv]

Tried new code this morning but 10sec sensors still unavailable after first restart. After that I went back to sqare one and started testing every release of hass-ams. 1.00 - working after several restarts of HA. 1.01 - working after several restarts of HA. 1.02 - working after several restarts of HA. 1.03 - 10sec sensors unavailable after first restart of HA. 1.04 - 10sec sensors unavailable after first restart of HA. 1.05 - 10sec sensors unavailable after first restart of HA. As said earlier, problems started when upgrading to 1.04 and at the same time I upgraded HA to 0.107.7. Prior to this I was running 0.107.6 for many days without any problems. While running 0.107.6 - I was in mine opinion running hass-ams version 1.03 because I always upgrade when new versions are available in HACS (if no breaking changes in release notes). When I look at the release dates of hass-ams, I see that 1.02 was released on march 20th and 1.03 on march 24th. In light of the above findings I can't explain this in any other way than that I must have skipped the 1.03 release...

[turbokongen]

Perfect! The change between 1.02 and 1.03 is the serial port chooser: pyserial.tools. It does note in their documentation that this might not work on all systems. https://pyserial.readthedocs.io/en/latest/tools.html

So the next question is this: What OS is run on your failing devices? @jgallis @jnrksv Mine two working are: Ubuntu 18.04LTS On python 3.8 Raspbian buster Linux 4.19.97-v7+ On python 3.7

[jnrksv]

I'm running HassOS 3.12 on an Intel NUC NUC7PJYH2.

[turbokongen]

I have pushed the latest code to test in missing_sensors branch. This is a revert of the changes made between v1.02 and v1.03 excluded the parser changes for aidon.

[Hellowlol]

If that was the bug you should add it back. You can use hass.async_add_executor_job to await sync functions.

[turbokongen]

I have some code ready if this is the bug. Just waiting for confirmation.

[jgallis]

I run hassos_ova-3.12.vmdk version on a VM host.

Tryed the latest from missing_sensors branch, but still no sensors after restart

[jnrksv]

Just finsihed testing latest. Same findings as jgallis.

[turbokongen]

Ok. I have now pushed a complete revert of the changes between v1.02 and v1.03. Please test. 🙏

[jnrksv]

Now I've tested latest "missing sensors branch". Sorry to say that 10sec sensors become unavailable after first restart. Just to be absolutely sure of my previous findings, I deleted "missing sensor branch" and than installed 1.02. Did several restarts and 10sec sensors are up and running each time. There has to be a differance between latest "missing sensors branch" and 1.02. Right?

[turbokongen]

Yes, there are differences, but: v1.02 is working v1.03 is not working So the bug or error lies in the change between v1.02 and v1.03. I will try tomorrow to install HassOs on a virtual machine and enable full debug in homeassistant to try and see what is going on.

[turbokongen]

Well... I have now also set a a HassOS instance on a vmware virtual machine on windows, but I can't even replicate the problem there. 😞 Running hassos_ova-3.12

[Hellowlol]

@turbokongen I'll try to fix this.

[Hellowlol]

Ok, i got a pr for this. I dont really know what was wrong as I started with the logging that was messed up (_LOGGER as a const) and just took things along the way. I think this should work every single time, i have rebooted ha 20 times, and so far it as worked 🤞

[jgallis]

I'm afraid I still have the same problem. All the sensors come in at first, but after the restart, everyone disappears. But with me it seems that the hourly sensors reappear when they come in after every hour.

[Hellowlol]

Did you test the PR i created? Oh, that was merged. Can you stop Home Assistant and make sure you remove the __pycache__ folder (inside ams and parser folder) and try again? @jnrksv Can you try aswell?

[jgallis]

Have got this tested properly this evening, and the sensors also work after several restarts. My problem is then solved. Thanks for the help.

[turbokongen]

Have got this tested properly this evening, and the sensors also work after several restarts. My problem is then solved. Thanks for the help.

Thank you for the feedback! :100: