Toparvion / analog

🔎 Flexible web-based real-time log viewer
MIT License
19 stars 5 forks source link

Support composite logs of Kubernetes pods #25

Open Toparvion opened 4 years ago

Toparvion commented 4 years ago

Currently AnaLog supports composite logs consisting of sources with fixed paths only. Any change to a source's path requires corresponding change to AnaLog configuration and restart. This makes AnaLog's usage with Kubernetes extremely inconvenient when it comes to including pods into a composite log because pods have dynamic names. Every time a pod starts (either by user's command or by Kubernetes itself) it gets a new name, so the composite log configuration becomes invalid. We need to provide a way for the configuration to express a non-strict path to a pod. This presumably should not allow to include arbitrary number of matching pods into a composite log, but rather single pod with flexible name. As a result, in case of multiple matching pods AnaLog should emit a warning or an error describing all the matching pod names (and the one that actually included into the composite log, if any). Or we should give a chance to explicitly point to a desired pod candidate among all matching names. In terms of configuration syntax it can look like:

choices:
  - group: STDOUT                                                   
    plainLogs:
      - path: k8s://deployment/my-deployment/pod/my-pod-*1
      - path: k8s://deployment/my-deployment/pod/my-pod-*2/container/main-container

, where *1 and *2 are placeholders to denote a dynamic part of the name and the order number of a candidate (assuming that the order itself doesn't matter).

Alternative solutions are welcome!

Toparvion commented 4 years ago

At this moment it seems that the optimal way to do this is to automatically expand the includes section of composite logs, e.g.:

    compositeLogs:
      - title: 'A cmoposite log'
        uriName: 'test-composite'
        includes:
          - path: k8s://deploy/some-deployment/*
            timestamp: yyyy-MM-dd HH:mm:ss,SSS

This should be treated by AnaLog as enumeration of all the pods comprising the some-deployment. This, however, leaves some questions:

  1. When the expansion should take place?
    The options are: on every choices building (may slow down responses to the clients) or just once on choices configuration loading (may lead to stale choices in case of pods re-creation).
  2. How to describe such a configuration from browser's address bar?
    The problem here is the timestamp that currently can not be passed through the address bar. One possible option here is to rely on the prefix that kubectl can provide (starting from v1.17) for every log line. But it requires a significant change to the whole logic of composite logs tailing as it heavy relies on timestamp notion. Let it be the next enhancement (aligned with the k8s v1.17 version spreading).
Toparvion commented 4 years ago

The work is paused due to the preparation for a talk and because of some discovered issues that need to be addressed before going on.