search

segmentio / ecs-logs

Log forwarder for services ran by ecs-agent.
MIT License
114 stars 16 forks source link

Question: Difference from awslogs driver #2

Closed shinzui closed 8 years ago

shinzui commented 8 years ago

When should you use ecs-logs now that the awslogs driver works well?

achille-roussel commented 8 years ago

For now the main (or only) difference is that the awslogs driver uses the container ID as a stream name where ecs-logs uses the container name. The latter is easier to read and reason about since it contains the ECS task number and unique ID generated by ECS.

If you have no need for it you can very well get rid of it. I'm working on improving ecs-logs and maybe add the ability to fan out to multiple log destinations or support structured (JSON) log events. We are still experimenting with logging and if you have ideas about what would be interesting to support or if you want to share what has worked well in your experience we'll be happy to hear about it!

achille-roussel commented 8 years ago

Another upside to using ecs-logs is it keeps logs available via docker logs because it pulls them from the journal. With the awslogs driver logs aren't available via the standard docker logs command. see https://docs.docker.com/engine/admin/logging/overview/

shinzui commented 8 years ago

Thank you for the answer. I was hoping that you would address the ability to fan out to multiple destinations because that's exactly what I'm looking for. The stream name is an issue for us so that's a good benefit and having the logs available via docker logs is useful, but the biggest problems is having the logs shipped outside of cloudwatch in an easy way. I see that you have a pull request to address that and I am looking forward to seeing it merged.

Out of curiosity, could you please share your logging stack at segment.

achille-roussel commented 8 years ago

We're using loggly and elasticsearch to store and consume our logs, we're also experimenting with cloudwatch as we speak (currently deploying ecs-logs to prod as part of this).

The reason we didn't include these tools in stack was that we wanted it to be an AWS-only solution and not require external dependencies to other third-party services so it can be useful to as many as possible and not specific to the Segment use-case.