segmentio / ecs-logs

Log forwarder for services ran by ecs-agent.
MIT License
114 stars 16 forks source link

[General] Support more than ECS based services #29

Open achille-roussel opened 8 years ago

achille-roussel commented 8 years ago

Currently the only limitation for only supporting ECS services is the way the journal reader extracts metadata to guess the log groups and streams. The log group is taken from the CONTAINER_TAG data and the log stream comes from CONTAINER_NAME. This works fine with ECS services which have a unique container name but for some services ran in production that's not the case, but we still want to have meaningful log groups and streams.

One way to address this is making the journal reader get both information from the CONTAINER_TAG parameter by supporting a richer format for the value. For example something like CONTAINER_TAG=group:stream would allow the docker run command to use log tags with a format like:

docker run ... --log-opt tag="{{.Name}}:{{.ID}}" ...

That would use the container name (likely a predictable value) as log group and the container ID as log stream.

One of the advantages of this approach is it can be implemented in a backward compatible manner, if the CONTAINER_TAG doesn't contain a : the program can rely on looking for the log stream name in CONTAINER_NAME.