search

atomita / fluent-plugin-aws-elasticsearch-service

This repository is extends of https://github.com/uken/fluent-plugin-elasticsearch/ which made connectable to Amazon Elassticsearch Service using Aws Signers V4. Please check the requirements to connect to Amazon Elassticsearch Service without using Aws Signers V4 for before starting use. (reference https://groups.google.com/forum/#!msg/fluentd/uW87VAOqxeE/cfhenicEBAAJ)
MIT License
122 stars 49 forks source link

Is shard configuration possible while creating index #69

Closed manojverma-net closed 3 years ago

manojverma-net commented 3 years ago

Question

Is there any property or something which I can use to setup index shard like no of shard for index.

Expected Behavior or What you need to ask

<match **>
   @type aws-elasticsearch-service
   @id out_aws_es
   @log_level debug
   include_tag_key true
   <endpoint>
     url "#{ENV['ELASTICSEARCH_ENDPOINT']}"
     region "#{ENV['ELASTICSEARCH_REGION']}"
   </endpoint>
   logstash_format 'true'
   index_name  'logstash'
   type_name 'fluentd'

   shard_count 1  <----- looking for this flag.

</match>
manojverma-net commented 3 years ago

@atomita @cosmo0920 - please provide your valuable input on this.

cosmo0920 commented 3 years ago

We won't plan to support it. We can specify it with number_of_shards on Elasticsearch index templates:

{
    "template" : "your-awesome-indices-*",
    "settings" : {
        "number_of_shards" : 1
    }
}

ref: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-template.html ref: https://github.com/uken/fluent-plugin-elasticsearch#index-templates