Is it possible to send data to aws cloudwatch loggroup?

[JackJiaJJ]

I tried to send logs from kafka to aws cloudwatch log groups, I searched some document, it mentioned the configuration something like below, but can NOT work well

output { cloudwatch_logs { log_group_name => "my-log-group" log_stream_name => "my-log-stream" region => "us-east-1" access_key_id => "my-access-key-id" secret_access_key => "my-secret-access-key" message => "%{message}" timestamp => "%{timestamp}" } }

I got below error:

[2023-05-15T02:23:24,356][ERROR][logstash.plugins.registry] Unable to load plugin. {:type=>"output", :name=>"cloudwatch_logs"} [2023-05-15T02:23:24,370][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (PluginLoadingError) Couldn't find any output plugin named 'cloudwatch_logs'. Are you sure this is correct? Trying to load the cloudwatch_logs output plugin resulted in this error: Unable to load the requested plugin named cloudwatch_logs of type output. The plugin is not installed."

[JackJiaJJ]

I tried I logstash 8.7.0 and 8.5.3, neither can work

[Anarhyst266]

@JackJiaJJ I can propose you to use my https://github.com/Anarhyst266/logstash-output-awslogs It's not well-documented, but there is an example:

output {
  awslogs {
    region => "us-east-2"
    access_key_id => "PLACEHOLDER"
    secret_access_key => "PLACEHOLDER"
    log_group_name => "/test_log_group"
    log_stream_name => "test_stream"
  }
}

It was created for my own needs, so you can't specify timestamp or message fields, it will use @timestamp as timestamp and the whole event in JSON format as message

To install it you can simply use logstash-plugin install logstash-output-awslogs