Unable to parse metadata

[SimplySeth]

Problem

2020-09-21 19:11:16 +0000 [warn]: #0 dump an error event: error_class=RuntimeError error="failed to expand `record[\"metadata\"][\"log_stream_name\"]` : error = undefined method `[]' for nil:NilClass" location="/usr/local/bundle/gems/fluentd-1.8.1/lib/fluent/plugin/filter_record_transformer.rb:310:in `rescue in expand'" tag="cloudwatch.in" time=2020-09-21 19:11:16.924052533 +0000 record={"log_message"=>"time=\"2020-09-

Steps to replicate

 <source>
    @type cloudwatch_logs
    @log_level "error"
    tag "cloudwatch.in"
    format /^(?<log_message>.*)/
    log_group_name "/aws/eks/mycluster/cluster"
    use_log_stream_name_prefix true
    region "us-west-2"
    state_file "/var/log/cloudwatch.in.state"
    include_metadata true
    <parse>
      @type regexp
      expression ^(?<log_message>.*)
    </parse>
  </source>
  <filter cloudwatch.in.**>
    @type record_transformer
    enable_ruby 
    <record>
      log_stream ${record["metadata"]["log_stream_name"]}
      component ${record["metadata"]["log_stream_name"].split('-')[0..-2].join('-')}
    </record>
  </filter>

Expected Behavior or What you need to ask

Log stream name and component name would show up in the logs

Using Fluentd and CloudWatchLogs plugin versions

• OS version Kubernetes 1.16

2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-cloudwatch-logs' version '0.9.4'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-concat' version '2.2.2'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-detect-exceptions' version '0.0.13'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '4.0.9'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-jq' version '0.5.1'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-kafka' version '0.13.0'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-kinesis' version '3.2.2'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-kubernetes_metadata_filter' version '2.4.6'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-prometheus' version '1.6.1'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.1.0'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.3.0'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-secure-forward' version '0.4.5'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-splunk-hec' version '1.0.1'
2020-09-21 19:11:08 +0000 [info]: gem 'fluent-plugin-systemd' version '0.3.1'
2020-09-21 19:11:08 +0000 [info]: gem 'fluentd' version '1.8.1'