search

pelias / terraform-elasticsearch

Terraform scripts for running an Elasticsearch cluster
10 stars 6 forks source link

high disk watermark more than flood stage disk watermark #8

Open missinglink opened 5 years ago

missinglink commented 5 years ago

saw this in the ES6 logs:

java.lang.IllegalArgumentException: high disk watermark [99%] more than flood stage disk watermark [95%]
orangejulius commented 5 years ago

Ugh ok. I guess this repo will have to handle setting the flood stage disk watermark as well, and instruct people to ensure the two values are set appropriately.

missinglink commented 5 years ago

Are those settings not currently coming from this repo?

orangejulius commented 5 years ago

They are configurable, so any terraform project using this repository can set them.

missinglink commented 5 years ago

I had a look into this and unfortunately, the flood_stage watermark isn't the same as the other watermark settings (high/low) in that it's not dynamically updatable, so presumably it needs to be set in elasticsearch.yml instead of updated via a curl request ☹️ 

curl -XPUT \
  -H 'Content-Type: application/json' \
  -d '
{
  "persistent": {
    "cluster.routing.allocation.disk.watermark.flood_stage": "50%"
  }
}' \
'localhost:9200/_cluster/settings?pretty=true'
{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "persistent setting [cluster.routing.allocation.disk.watermark.flood_stage], not dynamically updateable"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "persistent setting [cluster.routing.allocation.disk.watermark.flood_stage], not dynamically updateable"
  },
  "status" : 400
}