aws / opsworks-cookbooks

Chef Cookbooks for the AWS OpsWorks Service
Other
1.06k stars 1.23k forks source link

Invalid log_stream_name created by opsworks_cloudwatchlogs template when wildcard logs used #350

Open bananastalktome opened 8 years ago

bananastalktome commented 8 years ago

If a wildcard is used to capture logs, the awslogs.conf.erb template creates a section with a log_stream_name that appears to be invalid in CloudWatch. The stream is not automatically created by the agent, and if I try to manually create the log stream using the CloudWatch web interface with the log_stream_name the template generates, I get the message "There was an error creating your Log Stream. Please try again."

The following is the custom JSON I used for the opsworks_cloudwatchlogs::default recipe:

{
  "opsworks":{
    "cloud_watch_logs_configurations":{
      "my-test-app":[
        {"file":"/var/log/httpd/*-access.log"}
      ]
    }
  }
}

The awslogs.conf that is created is:

[general]
state_file = /var/lib/awslogs/agent-state

[my-test-app /var/log/httpd/*-access.log]
datetime_format = [%Y-%m-%d %H:%M:%S]
log_group_name = my-test-app
log_stream_name = rails-app1 - /var/log/httpd/*-access.log
initial_position = start_of_file
file = /var/log/httpd/*-access.log

If I remove the wildcard from the log_stream_name and restart the awslogs service on the instance, logging is captured by CloudWatch as expected.

EDIT: After further consulting the guide (https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/AgentReference.html), it seems wildcard files are not intended to be used as I stated in my examples, but instead only for a series of files of the same type. Nonetheless, the issue still remains in cases where something like /var/log/httpd/access.log-* is used (assuming rotated logs are not gzipped).

richseviora commented 8 years ago

The log_stream_name does not look valid. Per your link, it should be a value and if the file being generated has spaces in the value for that attribute I imagine it'll error out as it gets interpreted as multiple values.

Have you tried specifying the log_stream_name as a string without spaces?

bananastalktome commented 8 years ago

It does not look like log_stream_name is a parameter accepted by the template - awslogs.conf.erb has log_stream_name = <%= @hostname %> - <%= log_path %>.