ugur99
commented
1 year ago Thank you for the quick reply!
Actually if default volumeMount/volume paths would be overridden, application can be more easily adapted for different use cases. Depending on the use case it may not needed to mount the default volumes that current helm chart forces. Promtail provides this kind of flexibility; and it is useful when using different log collector instances for different purposes by isolating them each other. Actually instead of breaking functionality would not it provides more flexibility?
the kubernetes_logs source in particular, it will only collect logs from /var/log/pods so adjusting the volume mounts would break the functionality of that source
If data_dir is used for preventing data duplication so using pvc would be more appropriate. Because using hostPath for persistency and write operations can not be an option. In the point of view of a k8s admin it is a critical security concern; and should not be allowed. But for the users who would be ok with some duplication logs after pod restarts it would not be so terrible thing depending on the data collected but definitely using pvc would be more appropriate if you are using such kind of checker. Promtail does not use/provide any persistency, so not sure how they can handle duplications.
Secondly using an emptyDir with a source like kubernetes_logs or file, would also essentially break functionality.
Currently we are using Promtail to collect pod logs, and we are trying to adopt vector for collecting some log files on the host like kube-apiserver audit logs and its a good opportunity for us to monitor its performance.
spencergilbert
Signed-off-by: Ugur Can Ozturk ugurozturk918@gmail.com
What type of PR is this?
/kind feature
What this PR does / why we need it:
Using hostPath as a data directory might not be best practice unless it has to be; I believe users who would like to use emptyDir instead of hostPath should be able to do that.
It would be great if mounted volumeMounts can be overridden by users based on the logs data path that they want to collect.
Another good feature would be changing log level easily
Which issue(s) this PR fixes:
As an alternative log collector we're trying to use vector hoping it is more lightweight and performative. But since it's trying to write mounted hostpath for the data-dir; and since as a cluster-wide policy we don't allow any write actions for the hostpath volumes; it could not run with current helm chart values. I'm hoping this PR essentially can solve this problem.
Ref: Kubernetes Documentation for the hostPath volumes