Toparvion
commented
5 years ago @gamefundas Sounds great👍🏼 I have not enough experience with these tools as log sources and therefore must evaluate their compatibility with AnaLog first. Generally speaking, current AnaLog architecture allows to relatively easy plug-in a source that matches the following conditions:
- it has a CLI client to interact with;
- the client can operate as a long-running process producing log lines to standard output;
- [optional but very useful] the client can deliver log events (such as rotation, appearance, etc) to error output.
Of course integration with other log sources can be implemented in other ways (say direct API communication) but it would require more changes to AnaLog itself.
For the time being AnaLog's integration layer is not stabilized yet so it is early to talk about full-featured plugin subsystem but I think that after implementation of 2-3 new integrations it would become clear enough how to expose it.
I'm going to try out some of aforementioned tools. What do you think, which one is most demanded?
gamefundas
With the ability to source logs from containers (docker, kubes etc), think it will be even more effective to support trailing log events from central log stores/shippers like AWS Cloudwatch, Redis, Logstash etc.
In case of cloudwatch the log group name and stream name can serve as input and the analog url can even be modeled to something like http://analog-host/cw/log-group/log-stream.
In case of redis, pubsub functionality can be used to subscribe log channels using a name pattern and the analog url can be something like "http://analog-host/redis/channel-name".
Static configurations, quite similar to plaintext can also be defined for these. The ground work for all of this I suppose is already out there, only clients will need to be written. In the long run a plugin model will be desirable where people can simply provide their own implementations of a Client interface and Analog simply initializes them.
This can prove to be so useful as the tool can be envisioned as a generic websocket based "Real-time stream/event viewer". A lot of different sources can be plugged in to view data in real time.