lost disk queue - After AKS Node Restarts

[zohebk8s]

Hi Team,

Yesterday we have restarted our AKS Nodes and we have lost the disk queue. We see a new folder being created at AKS Node.

Here is the configuration file for filebeat:

volumeMounts:

• name: config mountPath: /etc/filebeat.yml readOnly: true subPath: filebeat.yml
• name: data mountPath: /usr/share/filebeat/data
• name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true
• name: varlog mountPath: /var/log readOnly: true volumes:
  • name: config configMap: defaultMode: 0600 name: filebeat-config
  • name: varlibdockercontainers hostPath: path: /var/lib/docker/containers
  • name: varlog hostPath: path: /var/log

    data folder stores a registry of read status for all files, so we don't send everything again on a Filebeat pod restart

  • name: data hostPath: path: /var/lib/filebeat-data type: DirectoryOrCreate

In order to retain the existing data/disk queue, is there a specific configuration we need to do in filebeat yaml?

As AKS node restart should not clear the disk queue and the data should be persistent even after AKS node restart.

[botelastic[bot]]

This issue doesn't have a Team:<team> label.

[botelastic[bot]]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!