424: Modify L0 API ECS instance user data to send logs to CloudWatch

[jparsons04]

What does this pull request do? The initial user data script that is executed at l0 api instance launch time will now stream 6 logs into their own log streams for the api instance's CloudWatch Log Group:

• /var/log/docker
• /var/log/ecs/cloudwatch-logs-start.log
• /var/log/ecs/ecs-agent.log
• /var/log/ecs/ecs-init.log
• /var/log/messages
• /var/log/user-data.log

How should this be tested? Create a new layer0 api instance and observe the new log streams created for your l0 instance.

Note: You need to have the fixes from issue #455 and #456 applied to run this successfully.

~Checklist~ ~- [ ] Unit tests~ ~- [ ] Smoke tests (if applicable)~ ~- [ ] System tests (if applicable)~ ~- [ ] Documentation (if applicable)~

closes #424

[zpatrick]

@jparsons04: Have you verified this works in AWS? If so, :shipit:

[diemonster]

The only thing preventing this from being merged is whether or not it's worth having the debug logs as a default (e.g. if they're too chatty). Or: making this flag configurable, which honestly seems a bit heavy-handed.

@jparsons04 if you could post a gist of what the logs look like with this flag enabled, it would be appreciated.

[jparsons04]

It's actually not enough to simply add these lines to the user data bash script. We also have to do a number of additional things to pipe the container's system logs to CloudWatch. This article has the basic idea, but as it stands, the changes I've made don't actually do anything and require additional work.

http://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_cloudwatch_logs.html See the section Configuring CloudWatch Logs at Launch with User Data

[sesh-kebab]

As long as DEBUG is a good default among crit | error | warn | info | debug then this is good 👍

[jparsons04]

After discussion, I think I'm going to turn down the logging levels to INFO for both the docker daemon and the ECS agent.

[zpatrick]

closing to reduce clutter