martomi / chiadog

A watch dog providing a peace in mind that your Chia farm is running smoothly 24/7.
MIT License
458 stars 120 forks source link

Chiadog fails processing log when `logrotate` rotates log file with `copytruncate` #289

Open flotti455 opened 3 years ago

flotti455 commented 3 years ago

Describe the bug

Instead of Chia's built-in log rotation, I use logrotate with the following configuration:

/home/chia/.chia/mainnet/log/*.log {
  su chia chia
  copytruncate
  rotate 9999
  nocompress
  missingok
  ifempty
  hourly
  datehourago
  dateext dateformat .%Y-%m-%d_%H-%M-%S
}

Whenever logrotate rotates Chia's log (as per my configuration, once per hour), Chiadog fails to process the log

[2021-09-07 20:03:27] [   DEBUG] --- Keep-alive check for HARVESTER: Last activity 201 seconds ago. (keep_alive_monitor.py:77)

I assume that Chiadog / pygtail has problems with the copytruncate rotation strategy.

Environment:

egeedot commented 1 year ago

I can also confirm we are having the same problem albeit this with Chia's default log rotation rather than logrotate. The symptoms are exactly the same, when the log rolls over Chiadog cannot handle the problem correctly and thinks that the harvester is offline.

Environment:

wrt Docker, I am using docker volume mounting with -v argument to directly mount the debug.log file into the container.