Open Toparvion opened 4 years ago
At this moment it seems that the optimal way to do this is to automatically expand the includes
section of composite logs, e.g.:
compositeLogs:
- title: 'A cmoposite log'
uriName: 'test-composite'
includes:
- path: k8s://deploy/some-deployment/*
timestamp: yyyy-MM-dd HH:mm:ss,SSS
This should be treated by AnaLog as enumeration of all the pods comprising the some-deployment
.
This, however, leaves some questions:
kubectl
can provide (starting from v1.17) for every log line. But it requires a significant change to the whole logic of composite logs tailing as it heavy relies on timestamp notion. Let it be the next enhancement (aligned with the k8s v1.17 version spreading).The work is paused due to the preparation for a talk and because of some discovered issues that need to be addressed before going on.
Currently AnaLog supports composite logs consisting of sources with fixed paths only. Any change to a source's path requires corresponding change to AnaLog configuration and restart. This makes AnaLog's usage with Kubernetes extremely inconvenient when it comes to including pods into a composite log because pods have dynamic names. Every time a pod starts (either by user's command or by Kubernetes itself) it gets a new name, so the composite log configuration becomes invalid. We need to provide a way for the configuration to express a non-strict path to a pod. This presumably should not allow to include arbitrary number of matching pods into a composite log, but rather single pod with flexible name. As a result, in case of multiple matching pods AnaLog should emit a warning or an error describing all the matching pod names (and the one that actually included into the composite log, if any). Or we should give a chance to explicitly point to a desired pod candidate among all matching names. In terms of configuration syntax it can look like:
, where
*1
and*2
are placeholders to denote a dynamic part of the name and the order number of a candidate (assuming that the order itself doesn't matter).Alternative solutions are welcome!