home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.49k stars 30.7k forks source link

Timer got out of sync. Resetting #7133

Closed frelev closed 6 years ago

frelev commented 7 years ago

Make sure you are running the latest version of Home Assistant before reporting an issue.

You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:

Home Assistant release (hass --version): 0.42-DEV

Python release (python3 --version):

Component/platform: Core

Description of problem: In the recent versions I have been getting this a lot in log.

17-04-16 03:55:26 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 04:17:37 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 04:20:42 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 04:52:41 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 05:04:35 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 05:47:22 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 06:07:44 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting

Expected: No entry in log.

Problem-relevant configuration.yaml entries and steps to reproduce:

Traceback (if applicable):

Additional info:

dpritt commented 7 years ago

Running 0.42.3 and getting a lot of these too, come in blocks of three about every 2 to 3 minutes.

17-04-16 21:47:28 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 21:47:30 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 21:47:32 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 21:49:05 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 21:49:15 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 21:49:18 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 21:49:33 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 21:49:35 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 21:49:40 WARNING (MainThread) [homeassistant.components.media_player] Updating sonos media_player took longer than the scheduled update interval 0:00:10
17-04-16 21:49:50 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 21:49:53 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 21:50:01 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-16 21:50:15 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting

Running on a Pi 2 bang up to date, with z-wave module. Running top shows that I am not hitting the CPU too hard, max of 80%

Regards

Dave

Nutti85 commented 7 years ago

I have the exact same issue on a Pi3 and I've had it for a while. It's flooding my log.

cmsimike commented 7 years ago

I have this same issue with hass installed in a VM

nottix commented 7 years ago

+1

nodomain commented 7 years ago

Also here:

Apr 20 22:06:17 tower hass[10946]: ERROR:homeassistant.core:Timer got out of sync. Resetting
...
Apr 20 22:14:31 tower hass[10946]: ERROR:homeassistant.core:Timer got out of sync. Resetting
Apr 20 22:15:00 tower hass[10946]: ERROR:homeassistant.core:Timer got out of sync. Resetting
Apr 20 22:18:11 tower hass[10946]: ERROR:homeassistant.core:Timer got out of sync. Resetting
Apr 20 22:24:22 tower hass[10946]: ERROR:homeassistant.core:Timer got out of sync. Resetting
Apr 20 22:26:32 tower hass[10946]: ERROR:homeassistant.core:Timer got out of sync. Resetting
jesperholstjh commented 7 years ago

Same here on pi3

frahlg commented 7 years ago

Same here, pi2. Hassbian image, all up to date.

nottix commented 7 years ago

I fixed deleting 350Mb of Database and removing History Statistics Sensor from configuration.

pbavinck commented 7 years ago

+1 on pi3

cmsimike commented 7 years ago

Given the hardware the errors happen on, it seems like any platform without an real time clock (RTC) has a problem? RasPis do not have an RTC, and I am not 100% sure on what my VMs do for their RTC

master-kenobi commented 7 years ago

same here on Pi3

irltankman commented 7 years ago

I have the same problem on a fresh install of Home Assistant 0.43.2 with only 2 media player's setup and presence detection through my netgear:

17-04-29 21:05:48 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
17-04-29 21:08:59 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
cmsimike commented 7 years ago

@irltankman where is hass installed on? what kind of machine?

irltankman commented 7 years ago

Raspberry Pi Model B (Rev 2.0, 512Mb)

irltankman commented 7 years ago

I have disabled my samsung TV's (my 6 series using websockets wasn't responding to commands from HA anyway) and the issue appears to have gone away. Issue with samungtv mediaplayer maybe?

ronnicek commented 7 years ago

Same here. RPI 3.

shortbloke commented 7 years ago

I see this message each time I select the History or Logbook on my RPi3 with 0.51.1

I had hoped adding samsung_tv to the ignore list for discovery would help things, but it hasn't

balloobbot commented 7 years ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment :+1:

PuckStar commented 7 years ago

0.55 still happening:

    Line 3210: 2017-10-18 23:17:32 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
    Line 9469: 2017-10-19 04:27:49 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
    Line 20047: 2017-10-19 12:52:13 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
    Line 30858: 2017-10-19 20:56:30 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
    Line 31097: 2017-10-19 20:56:33 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
fanaticDavid commented 7 years ago
2017-10-20 23:03:58 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
2017-10-20 23:04:47 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting

The first line was caused by loading the History page. The second line was caused by loading the Logbook page. Running Home Assistant 0.55.x with an external MariaDB (MySQL) instance for the recorder.

CCOSTAN commented 6 years ago

0.57.2 - Still happening.

Nov 18 05:03:02 Carlo-Pi hass[31296]: 2017-11-18 05:03:02 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
Nov 18 05:03:52 Carlo-Pi hass[31296]: 2017-11-18 05:03:52 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
Nov 18 05:04:04 Carlo-Pi hass[31296]: 2017-11-18 05:04:04 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
ruimarinho commented 6 years ago

0.58 - Still happening on a Synology NAS running the official home assistant image on Docker.

marcelveldt commented 6 years ago

Same issue on my Synology NAS running the official docker image. I notice slow responds from Home Assistant (up to 30 seconds delay in any automations for example) and very high memory usage (4GB in use). Simple reset of the docker container solves it but it can happen again anytime. I'll try to do some more investigating but so far I did not see anything

ruimarinho commented 6 years ago

I'm not experiencing high memory usage but rather high CPU usage (60%+).

talondnb commented 6 years ago

Seeing this on 0.58.1 on a Pi 3, AIO install.

AdrienHunter commented 6 years ago

Same for me using 0.58.1 using Pi3 B using Debian 9.1 Stretch & AIO install:

2017-11-25 01:35:28 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
2017-11-25 01:36:39 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
2017-11-25 01:37:00 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
andriej commented 6 years ago

Happened for me tonight, for first time. 0.58.1 on Debian / Orange Pi

luisfocosta commented 6 years ago

Same thing with a RPI3 native installation, Linear HUSBZB-1 (zwave usb controller), one zwave device configured, 4 chromecast, 1 apple tv, Plex, mqtt sensors, NEST thermostat, and dd-wrt presence.

KamremK commented 6 years ago

I got the samme issue when i was setting a timer for my switch. I used "wait"and it started filling the log. When i changed to "delay" instead the log errors seems to have stopped?

PI3 0.59.2 HassIO

CCOSTAN commented 6 years ago

I get a LOT of these errors. I do use wait_template a lot in my automation as well. Wonder if there is a correlation. The RPI is getting increasingly slow.. noticeably slow when performing actions. Hopefully some headway can be made with this issue.

0.59.2

ciotlosm commented 6 years ago

I just started getting these errors after migration to 0.60. I previously seen these when I tried to enable BLE tracker (reason why I didn't use it), but now it's back. Also "climate" component is now responding slower "Setup of climate is taking over 10 seconds.".

I think 0.60 degraded something for RPI users. I use RPI 3 with HassIO.

nordlead2005 commented 6 years ago

eh, it isn't just RPI users, I'm running Ubuntu and this came out of nowhere. I've added nothing to my configuration since 0.50 and this started happening. I took out my chromecast thinking maybe discovery of it was causing the problem and I woke up this morning to my lights taking 30 seconds to turn on after I entered the room.

marcelveldt commented 6 years ago

True, I'm running Hass om a pretty powerful NAS and it's happening to me too. Cpu and memory usage go high and the whole house is responding slow. Happens to me now 3 times a day so making it a but unusable. Currently I'm on a debug hunt. The issue is perfectly reproducible if you add some long running call in a module/component so I'm pretty sure the culprit is somewhere in a component misbehaving and/or combination with automations.

I'm currently testing by enabling/disabling components one-by-one until I find which one is blocking. Bit time consuming but so be it. I'd suggest others to do the same, after all this is a community driven product, right.

shortbloke commented 6 years ago

I can trigger it, just by opening history or logbook.

zxjack commented 6 years ago

0.60 2017-12-22 00:23:41 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting

FigJam23 commented 6 years ago

Same for me driving me up the wall moved from Hassio zero issues to Hassbian moving my config over system becomes laggy unstable automation won’t run or take a long time to do so. I’m on hassbian 0.60 Lots of random timer out of sync errors

ciotlosm commented 6 years ago

I think I found the problem with mine. I was getting these errors starting with 0.60.0.

When I upgraded to 0.59 I added:

frontend:
  javascript_version: latest

When I upgraded to 0.60 I removed it because of a documented issue with ios and I had:

frontend:

This caused some other problems as well as it seems it parses User-Agent to determine what your browser supports so now I moved to:

frontend:
  javascript_version: es5

After using the last config I noticed all the Timer got out of sync. Resetting disappeared.

I'm not sure if it's the same for everyone, but for me it solved the problem.

Edit 1: It seems there are two other issues that might be linked to this one and the performance problem #11201 and #11234

PS: I'm running HassIO on a Raspberry PI 3 I'm using bluetooth scanner (not the BLE one) I've removed MiFlora since last version as that caused other problems (waiting for a fix that is in pipeline) My configuration is using packages (packages: !include_dir_named packages) I have ios phones and I'm using xiaomi_aqara devices

CCOSTAN commented 6 years ago

I wish this fixed mine but it didn’t. :(

.

Check out my virtualization blog at www.vCloudInfo.comhttp://www.vCloudInfo.com! Follow me : Twitter: @ccostan | FaceBook.comhttp://FaceBook.com\VMwareInfo

On Dec 26, 2017, at 2:52 PM, ciotlosm notifications@github.com<mailto:notifications@github.com> wrote:

I think I found the problem with mine. I was getting these errors starting with 0.60.0.

When I upgraded to 0.59 I added:

frontend: javascript_version: latest

When I upgraded to 0.60 I removed it because of a documented issuehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fhome-assistant%2Fhome-assistant%2Fissues%2F11234&data=02%7C01%7CCarlo.Costanzo%40ipm.com%7Cbae5993a8acc4ad7813508d54c9a3a86%7C192fb3aaa4a64b9992a744a240ffed4f%7C0%7C0%7C636499147663130730&sdata=5BbWsrpYD27jfgyMxonVdZ6WzWebcXxpccos1ZxwLV0%3D&reserved=0 with ios and I had:

frontend:

This caused some other problems as well as it seems it parses User-Agent to determine what your browser supports so now I moved to:

frontend: javascript_version: es5

After using the last config I noticed all the Timer got out of sync. Resetting disappeared.

I'm not sure if it's the same for everyone, but for me it solved the problem.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fhome-assistant%2Fhome-assistant%2Fissues%2F7133%23issuecomment-354006633&data=02%7C01%7CCarlo.Costanzo%40ipm.com%7Cbae5993a8acc4ad7813508d54c9a3a86%7C192fb3aaa4a64b9992a744a240ffed4f%7C0%7C0%7C636499147663130730&sdata=4o8dB1Af4dyBa2Hnq7g5L%2Fzc%2FNq3IYeVw%2BGSSOXvg78%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACD3NGIdsz4caGh-tMWKJnhHr1AhDM57ks5tEU6JgaJpZM4M-llr&data=02%7C01%7CCarlo.Costanzo%40ipm.com%7Cbae5993a8acc4ad7813508d54c9a3a86%7C192fb3aaa4a64b9992a744a240ffed4f%7C0%7C0%7C636499147663130730&sdata=RaPr%2FB%2Fi%2BYq4DszQi27IChMAO4atxG60OuWh6A2tooM%3D&reserved=0.

net4unner commented 6 years ago

Same issue here with .60

2017-12-28 14:50:53 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
2017-12-28 15:27:37 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
2017-12-28 15:29:54 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
2017-12-28 15:37:47 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
2017-12-28 16:09:55 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
2017-12-28 16:29:18 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
2017-12-28 19:30:05 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting
naamah75 commented 6 years ago

Same issue with 0.58, 059 and 0.60 HASS.io on Raspberry Pi 3. Tryed also with a ultra fast SD, the problem is come out also after a clean installation.

nordlead2005 commented 6 years ago

@CCOSTAN I tried that, and I thought it worked, but at 10PM last night the Timer got out of sync errors came back (so, it lasted a good bit longer than before).

mar-schmidt commented 6 years ago

Ive got the same output to the logfile running on rp3. Seems like the issue appears more often the more components I'm adding

bieniu commented 6 years ago

This is not an ideal solution but I have much less of these errors after removing the snmp sensors from configuration.

HelloFromEarth commented 6 years ago

I'm getting the same error with 60.1 on a pi3. It seems to be related to calling a shell_command: play_sound: sudo omxplayer /usr/share/sounds/alsa/sound.mp3

Kallb123 commented 6 years ago

I'm getting the error on 61.1. Not sure what the cause is in my setup. Can the error be any more detailed?

marcelveldt commented 6 years ago

The issue will happen if one or more components hold up the event loop. So they're doing some stuff that takes time while it should't. In that case the event loop gets out of drift resulting in these errors. I was able to completely fix the issues by checking each and every script, automation and component for errors and prevent stuff from long running calls/actions. Also watch out if you're using Homebridge because I have seen multiple times that one was causing the issue when started before hass itself. A simple restart of homebridge after hass start/restart fixed it and making sure Homebridge only receives updated for components that are really needed.

Kallb123 commented 6 years ago

@marcelveldt what's the best way of tracking down the culprit? I don't exactly know how long each automation/script/service will take.

marcelveldt commented 6 years ago

Well, I just went into trouble of disabling all stuff and enable it one-by-one. For me it was a combination of Homebridge and some advanced automations/scripts I wrote. In the process I came across a few modules that are not properly async written but it seems that hass takes care of wrapping that into some async helpers itself.

ryanwinter commented 6 years ago

I am seeing this problem in 0.62 running on a freebsd box. Causing my zha service to flake out which requires a restart :|

Seems to have gotten worse in 0.62 so will likely downgrade, or maybe start disabling components to try and figure out which one is causing the problem.

master-kenobi commented 6 years ago

I started with fresh installation of 0.62.1 and I still receive...

2018-02-03 07:52:24 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting

There is no added components, no changes to the configuration file. Just fresh installation.