level=error ts=2021-05-12T09:38:46.447252922Z caller=filetarget.go:276 msg="failed to tail file, stat failed" error="stat /var/log/pods/seldon-analytics_seldon-analytics-prometheus-node-exporter-c7xmn_bc61dab4-450f-4540-a145-1ed57ec9bfa5/prometheus-node-exporter/2.log: no such file or directory" filename=/var/log/pods/seldon-analytics_seldon-analytics-prometheus-node-exporter-c7xmn_bc61dab4-450f-4540-a145-1ed57ec9bfa5/prometheus-node-exporter/2.log
I'm seeing errors like these in promtail output. If file doesn't exist because promtail was down for extended amount of time then just log the fact and drop it from tracked files and don't remind me about it every 10 seconds forever.
Something like this also polutes error logs:
level=error ts=2021-05-12T08:49:29.705845785Z caller=filetarget.go:264 msg=" failed to remove directory from watcher" error="can't remove non-existent inotify watch for: /var/log/pods/polyaxon_plx-bb3f99e9354642d0b20c17301b50b62c-master-0_c26e02f1-425c-447b-8944-23816ea18873/polyaxon-init-job"
Is this really an error? What can I do about it?
This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.
Describe the bug
level=error ts=2021-05-12T09:38:46.447252922Z caller=filetarget.go:276 msg="failed to tail file, stat failed" error="stat /var/log/pods/seldon-analytics_seldon-analytics-prometheus-node-exporter-c7xmn_bc61dab4-450f-4540-a145-1ed57ec9bfa5/prometheus-node-exporter/2.log: no such file or directory" filename=/var/log/pods/seldon-analytics_seldon-analytics-prometheus-node-exporter-c7xmn_bc61dab4-450f-4540-a145-1ed57ec9bfa5/prometheus-node-exporter/2.log
I'm seeing errors like these in promtail output. If file doesn't exist because promtail was down for extended amount of time then just log the fact and drop it from tracked files and don't remind me about it every 10 seconds forever.
Something like this also polutes error logs:
level=error ts=2021-05-12T08:49:29.705845785Z caller=filetarget.go:264 msg=" failed to remove directory from watcher" error="can't remove non-existent inotify watch for: /var/log/pods/polyaxon_plx-bb3f99e9354642d0b20c17301b50b62c-master-0_c26e02f1-425c-447b-8944-23816ea18873/polyaxon-init-job"
Is this really an error? What can I do about it?