amazon-archives / logstash-input-dynamodb

This input plugin for Logstash scans a specified DynamoDB table and then reads changes to a DynamoDB table from the associated DynamoDB Stream.This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline. This gem is not a stand-alone program.
Apache License 2.0
105 stars 27 forks source link

How can I stop "type" in my dynamo table being used as _type in my index? #32

Open mikebrules opened 7 years ago

mikebrules commented 7 years ago

Hello, I have a "type" field in my data that I DON'T want to be mapped to the index - that is a different field (called assetType). Logstash seems to be taking my "type". I know I could override that by setting document_type in my output, but I have two tables I want to map into two different Elastic indexes in my output - like this;

    output {
      if[type] == 'story' {
        elasticsearch {
          hosts => ["10.0.2.15:9200"]
          index => "storys"
          document_type => "%{assetType}"
          document_id => "%{assetId}"
       }
   }
   stdout {}
  }

but I want to do this (use assetType)

  output {
      if[assetType] == 'story' {
        elasticsearch {
          hosts => ["10.0.2.15:9200"]
          index => "storys"
          document_type => "%{assetType}"
          document_id => "%{assetId}"
       }
   }
   stdout {}
  }

.....but I can't do that, as type is being used automatically, and is the only field I can use in my input....

Does anyone know how I can tell logstash the "type" field is actually my assetType value?