let ec2 instances log somewhere other than only on the instance

[zguillen]

We need to be able to look at logs of failed ec2 instances or containers in order to troubleshoot when failures happen. Currently logs are only available via attaching to the ec2 instance while it's still running. Once the instance is terminated we lose all logs.

research what others are doing WRT logging... our ideas

1. write to the mounted efs drive (seems easy?) but would have to implement our own log rotation/archiving strategy
2. log to cloudwatch? but man those logs are a pain to read

[zguillen]

add to logging

• who is running the notebook (currently just the user_id jupyter hub creates but later might be msd-live user id)
• what container is running