Constant "Timer got out of sync" errors

[danodemano]

Home Assistant release (hass --version): 0.39.0

Python release (python3 --version): 3.4.2

Component/platform: Core

Description of problem: Constant ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting Errors in log file

Expected: No error

Problem-relevant configuration.yaml entries and steps to reproduce: N/A just happens

Traceback (if applicable): N/A

Additional info: Wasn't happening in 0.38.3. No real consistency to timing:

17-02-26 14:31:24 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:31:26 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:31:29 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:31:31 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:31:35 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:31:37 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:31:40 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:31:45 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:31:52 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:32:00 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:32:03 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:33:31 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-02-26 14:35:30 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting

[balloob]

Yeah, my suspicion is that Z-Wave is doing some I/O somewhere on the event loop…

[philhawthorne]

I'm wondering if it could be related to the number of components people are using, and how busy they are?

Z-wave would send a lot of updates, so Z-wave and a couple of other components could start causing this by themselves. But for users who don't have Z-wave, but might have 20 little components enabled might see these errors with lots of things trying to push updates through.

Just an idea.

[balloob]

I actually found Z-Wave doing I/O in the event loop https://github.com/home-assistant/home-assistant/pull/6369

[philhawthorne]

Bam!

[pvizeli]

@philhawthorne i have test that my hass on rpy3 can process 20'000 events/state in 1 minute.

But if a component/platform do bad stuff inside loop, it give a bad expirence for user and block the hole core. So we add this message to find out which platform make troubles.

[arsaboo]

There are also lot of duplicate media_player entities in 0.39 (https://github.com/home-assistant/home-assistant/issues/5588). Wondering if that may be slowing things down.

[cgarwood]

Not sure if related, but since upgrading from 0.38.3 to 0.39.2 when my Aeotec multisensor or home energy meter updates, my HASS console gets spammed with close to 200 state changed events (which also get saved to the home assistant database). All of them are duplicates of each other.

Just upgraded to 0.39.3 to see if that helps.

[cgarwood]

0.39.3 did not fix it for me. Example of what I'm seeing in the console: https://hastebin.com/ojifedoqog.pl That was all I could copy before I couldn't scroll my console up anymore. It created around 200 state changed events that were logged into the home-assistant database. In the middle of that I saw a few red lines flash by which upon looking at the HA log I see were Timer out of sync errors

[balloob]

@cgarwood that is not related and is also expected https://github.com/home-assistant/home-assistant/pull/5844

[pr3sidentspence]

I am seeing these in 0.39.3 as well. I am seeing both hass and homebridge running 30-40% CPU each on my Rpi model B. I don't remember seeing this before.

[n5qm]

I am still seeing this error on 0.39.3.

I have narrowed it down to only happening when I activate the bluetooth_le_tracker component. If I disable that component the message goes away.

[tsvi]

Seeing this error with 0.39.3:

The following are the components I have:

homeassistant frontend config http updater discovery conversation history logbook sun sensor: platform: yr tts: platform: google vera microsoft_face ffmpeg camera

[Toonix]

Downgraded to 0.38.4 finaly its back to 1% CPU and i can Switch light over Webinterface. With 0.39.3 it was slow and the switches didnt work most of the time. Plus the crashes when i used logbook

[balloob]

@Toonix such reports are only valuable if you include the platforms/components that you use.

[slimatic]

Broken for me as well on 0.39.3. Discovery no longer shows my media devices. It was working fine until I upgraded to 0.39.3 from 0.38. I have zwave component working and I do get the time out as well.

I am running default configuration settings plus zwave on hassbian (raspi2).

Reverted back to 0.38.4. Working as expected now.

[demonspork]

I am encountering the same issues, [homeassistant.core] Timer got out of sync. Resetting and very slow responses to any sort of api call including alexa and emulated_hue, z-wave is barely functioning. I have disabled discovery and my only new platform flux_led but neither had any effect.

EDIT: Reverting to 0.38.4 fixes it.

Currently on 0.39.3 Components/platforms: homeassistant http emulated_hue pushbullet html5push sun logbook discovery history recorder frontend plex hdmi_cec hue flux_led input_select input_slider input_boolean template switch wink zwave alexa automation template sensor darksky speedtest

[tethyso]

Same problem here since 0.39 (now on 0.39.3) Sometimes it takes 2-3 mins to just restart home assistant or reboot. Components/platforms: mqtt automation netatmo pushbullet bluetooth tracker kodi qnap dht

[pvizeli]

@tethyso can you retry with disable Bluetooth tracker?

[terrycarlin]

I disabled the Bluetooth tracker and the “Timer got out of sync” error messages seemed to have gone away. It might be my imagination but it seems like HA starts up faster also.

Thanks!

On Mar 7, 2017, at 8:44 AM, Pascal Vizeli notifications@github.com wrote:

@tethyso https://github.com/tethyso can you retry with disable Bluetooth tracker?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/home-assistant/home-assistant/issues/6253#issuecomment-284759390, or mute the thread https://github.com/notifications/unsubscribe-auth/AA1W1IM-ofzwPiWMoCtnD3k-MeH7RjMtks5rjXt2gaJpZM4MMfVC.

[pvizeli]

@balloob ^^ https://github.com/home-assistant/home-assistant/issues/4442

[demonspork]

Does this mean that there is something that bluetooth tracker and some other components does that causes this error because of a change to the core in 0.39? I am going to assume they are doing something bad in the first place.

[tethyso]

@pvizeli Disabled blueooth tracker but unfortunatly my restarting time did not improve: hass just hangs there for a couple of minutes before doing anything. Will monitor if I get more timer got out of sync logs, though!

[pvizeli]

@tethyso can you try with last dev and comment component out until you find the trouble maker?

It look like your blueooth are not deactivate, that will block the device_tracker a long time...

[wmn79]

Still have this issue after upgrading from 39.1 to 39.3. Components:

• Hue
• Z-Wave
• Sonos
• Chromecast
• Weatherunderground
• AsusWRT
• Harmony Hub
• Speedtest
• HP ILO

[flounderscore]

I got that error message, too. In my case the device_tracker.bluetooth_le_tracker was the culprit. Unless I set the scan_duration to 1 I get the out of sync error message every ~11 seconds. It did detect my devices though.

[wmn79]

I don't use the bluetooth device_tracker, so that isn't the cause in my case.

[pvizeli]

@flounderscore can you try to modify this line: https://github.com/home-assistant/home-assistant/blob/dev/homeassistant/core.py#L55 and change the pool size for test to 25?

Also do scan_duration set 60 or more. A to low value can overtask it (fix in dev).

[flounderscore]

@pvizeli Neither of the changes helped. Setting scan_duration to 60 causes the startup to take forever and the out-of-sync message to only appear every 60s.

[pvizeli]

Okay it is the bluetooth device that make the trouble. If you have a raspberry, please make a rpi-upgrade and apt-get update. Looks like that fix also troubles

[pr3sidentspence]

I'm getting these and don't have Bluetooth. It is a Pi though, and I'll try those commands.

On Fri, Mar 10, 2017 at 1:13 AM Pascal Vizeli notifications@github.com wrote:

Okay it is the bluetooth device that make the trouble. Uf you have a raspberry, please make a rpi-upgrade and apt-get update. Looks like that fix also troubles

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/home-assistant/home-assistant/issues/6253#issuecomment-285595672, or mute the thread https://github.com/notifications/unsubscribe-auth/AArbCaO1W0yfCHyLhaqNt9qXIQpStZ5eks5rkPg0gaJpZM4MMfVC .

[KptnKMan]

@pvizeli & @pr3sidentspence Upgraded my firmware and saw a definite uptick in responsiveness afterward.

Note that I downgraded to 0.38.4 first, and don't use blutooth on my RPi3, but I have been experiencing the issues noted in this thread when I upgraded to 0.39.3. I plan to upgrade to 0.39.3+ again in the coming weeks.

For anyone wondering, its pretty straightforward:

sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get install rpi-update
sudo rpi-update

[demonspork]

I have been having these issues and I am using Docker version, so not rpi specific problems.

On Fri, Mar 10, 2017 at 8:19 AM, Kareem Straker notifications@github.com wrote:

@pvizeli https://github.com/pvizeli & @pr3sidentspence https://github.com/pr3sidentspence Upgraded my firmware and saw a definite uptick in responsiveness afterward. Note that I downgraded to 0.38.4 first, but I plan to upgrade to 0.9+ again in the coming weeks. For anyone wondering, its pretty straightforward:

sudo apt-get update sudo apt-get upgrade -y sudo apt-get install rpi-update sudo rpi-update

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/home-assistant/home-assistant/issues/6253#issuecomment-285680129, or mute the thread https://github.com/notifications/unsubscribe-auth/AAaYsdA0iK11zsaCNgcqGHActWtqJkKcks5rkVwPgaJpZM4MMfVC .

[jamieshaw]

Same issues here and not using Docker, Bluetooth or rPI. Running natively on macOS.

[wmn79]

I have a RPI 3 but after doing apt-get update && apt-get upgrade, I still have these messages in the log. Doing rpi-update also makes no difference unfortunately.

[arsaboo]

@wmn79 Are you on 0.39.4?

[wmn79]

@arsaboo I am on 0.39.3. Not aware of a 0.39.4 version yet.

[arsaboo]

@wmn79 Sorry....0.39.3 is the latest....got carried away :)

[CCOSTAN]

I'm definitely getting these after my ZWAVE stuff loads up. Then they are non stop. (EVERY 5 SECONDS) This is on a RPI3 with 0.40.DEV (and 39.3).

Mar 11 02:51:59 Carlo-Pi hass[685]: 17-03-11 02:51:59 INFO (MainThread) [homeassistant.components.switch] Setting up sw
itch.zwave
Mar 11 02:51:59 Carlo-Pi hass[685]: 17-03-11 02:51:59 INFO (MainThread) [homeassistant.components.switch] Setting up sw
itch.zwave
Mar 11 02:51:59 Carlo-Pi hass[685]: 17-03-11 02:51:59 INFO (MainThread) [homeassistant.components.switch] Setting up sw
itch.zwave
Mar 11 02:52:00 Carlo-Pi hass[685]: 17-03-11 02:52:00 INFO (MainThread) [homeassistant.helpers.script] Script voice_not
ify: Executing step call service
Mar 11 02:52:12 Carlo-Pi hass[685]: 17-03-11 02:52:12 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Re
setting
Mar 11 02:52:15 Carlo-Pi hass[685]: 17-03-11 02:52:15 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Re
setting
Mar 11 02:52:16 Carlo-Pi hass[685]: 17-03-11 02:52:16 WARNING (Thread-11) [homeassistant.components.zwave] zwave not re
ady after 30 seconds, continuing anyway
Mar 11 02:52:16 Carlo-Pi hass[685]: 17-03-11 02:52:16 INFO (Thread-11) [homeassistant.components.zwave] final network s
tate: 5 Driver initialised
Mar 11 02:52:16 Carlo-Pi hass[685]: 17-03-11 02:52:16 INFO (Thread-11) [homeassistant.components.zwave] zwave polling i
nterval set to 60000 ms
Mar 11 02:52:18 Carlo-Pi hass[685]: 17-03-11 02:52:18 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Re
setting
Mar 11 02:52:22 Carlo-Pi hass[685]: 17-03-11 02:52:22 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Re
setting
Mar 11 02:52:25 Carlo-Pi hass[685]: 17-03-11 02:52:25 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Re
setting
Mar 11 02:52:29 Carlo-Pi hass[685]: 17-03-11 02:52:29 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Re
setting
Mar 11 02:52:34 Carlo-Pi hass[685]: 17-03-11 02:52:34 INFO (MainThread) [homeassistant.loader] Loaded wemo from homeass
istant.components.wemo
Mar 11 02:52:36 Carlo-Pi hass[685]: 17-03-11 02:52:36 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Re
setting
Mar 11 02:52:41 Carlo-Pi hass[685]: 17-03-11 02:52:41 INFO (MainThread) [homeassistant.components.light] Setting up lig
ht.hue
Mar 11 02:52:41 Carlo-Pi hass[685]: 17-03-11 02:52:41 INFO (MainThread) [homeassistant.components.light] Setting up lig
ht.hue
Mar 11 02:52:41 Carlo-Pi hass[685]: 17-03-11 02:52:41 INFO (MainThread) [homeassistant.components.light] Setting up lig
ht.hue
Mar 11 02:52:41 Carlo-Pi hass[685]: 17-03-11 02:52:41 INFO (Thread-5) [homeassistant.components.light.hue] Emulated h

[CCOSTAN]

These all went away with the suggestion of the PI firmware. rpi-update solved the timer issues for me.

https://twitter.com/home_assistant/status/839940982900563969

[heathpitts]

This firmware update seems to have fixed any timer issues I was having as well

[tethyso]

The firmware update solved timer issues and - at least for me - out of memory issues with mqtt. Somehow it broke my DHT (connect to rpi GPIOs) and Netatmo sensors....which both always worked. Will investigate on this. Thanks guys!

[EleanorRagone]

Running the firmware update did not solve the timer issues for me.

For what it's worth, I'm also having a lot of problems getting HA to actually poll my zwave devices (as in, I have polling_intensity: 1 and polling_interval: 5000 set on my thermostat, but there won't be a single log message from it for hours). Not sure if it's related.

[yajrendrag]

i am not running on a pi... i get 1 Timer got out of sync. Resetting error usually every day - coincident with zwave reset. fwiw.. my integrations are in previous comment in this thread above.. a running 0.39.3.

[Toonix]

For me it's much better. Running 0.39.3 and it feels now ok. The CPU is now about 4%. Log is reporting the usual chromcast errors, but till now no out of sync

[Toonix]

Ok it's Back. But it still better. Now I get this errors but it is at least working.

[slvwolf]

I hit this problem as well and upgrading the firmware or software did not help. On startup HA simply hit 100% CPU and kept repeating log line Timer got out of sync. Resetting. HA UI was not available and any automation scripts could not connect to the HA API.

I managed to solve this problem by simply removing one Z-Wave device from my network. The problem was quite obvious when checking the output logs from OpenZWave Control Panel. Line below was being spammed non-stop:

2017-03-11 22:14:12.004 Detail, 
2017-03-11 22:14:12.069 Detail, Unsolicited message received while waiting for ACK.
2017-03-11 22:14:12.069 Detail, Node005,   Received: 0x01, 0x15, 0x00, 0x04, 0x00, 0x05, 0x0f, 0x32, 0x02, 0x21, 0x64, 0x00, 0x00, 0x1a, 0x58, 0x2e, 0xc8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x35

Simply unplugging the node005 fixed the issue.

[danodemano]

This issue seems to be fixed in 0.40.0. I've been running for ~30 minutes with no out of sync errors. Though now I have issues with startup. :/ Anyway if everything still looks good by morning I'll close this out.

[EleanorRagone]

FYI this is still happening for me on 0.40.0 =(

[jamieshaw]

Same. Also seeing this error under 0.40.0 on macOS.

[kirichkov]

I've updated openzwave to 0.32 and homeassistant to 0.40.0 and now CPU usage is normal. I'll report back if this issue reappears after hass is running for some time.