unknown key [number_of_shards] for create index

[swimmadude66]

General information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
• ElasticHQ Version: 3.5.0 (docker)
• Elasticsearch Version: 7.0.0
• Python version: N/A
• Browser Vendor and Version (if applicable): Chrome 73

Issue Description

Create index fails (though it shows "index created successfully") with error: unknown key [number_of_shards] for create index elastichq

It appears that elasticHQ is passing

"settings" :  {
     "number_of_shards": 1, 
      "number_of_replicas": 1
}

instead of

"settings" : {
        "index" : {
            "number_of_shards" : 1, 
            "number_of_replicas" : 1 
        }
    }

Source Code / Logs

elastic-hq   | 2019-04-15 15:40:07,092   WARNING         elasticsearch   base.log_request_fail:97        PUT http://search:9200/testindex [status:400 request:0.004s]
elastic-hq   | 2019-04-15 15:40:07,093   DEBUG   elasticsearch   base.log_request_fail:105       > {"number_of_replicas":0,"number_of_shards":0}
elastic-hq   | 2019-04-15 15:40:07,093   DEBUG   elasticsearch   base.log_request_fail:110       < {"error":{"root_cause":[{"type":"parse_exception","reason":"unknown key [number_of_shards] for create index"}],"type":"parse_exception","reason":"unknown key [number_of_shards] for create index"},"status":400}
elastic-hq   | 2019-04-15 15:40:07,094   ERROR   elastichq       exceptions._request_wrapper:37          Oops! Something bad happened.

[royrusso]

Thanks. Looking at it - looks like the API changed in 7.x

[royrusso]

So this is confusing, as according to their docs, it should work: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#create-index-settings

[swimmadude66]

yeah, I saw that too. I'm willing to bet there's a config somewhere which disables "strict" validation on API bodies, but I'm posting here because this error occurs straight out of the box with latest elastic and latest elastichq. here's the docker compose file I am using to launch it, nothing fancy.

version: '3'

services:
  elastichq:
    image: 'elastichq/elasticsearch-hq:release-v3.5.0'
    container_name: 'elastic-hq'
    ports: 
      - '5000:5000'
    environment: 
      - HQ_DEFAULT_URL=http://search:9200
    links: 
      - search
    depends_on: 
      - search
  search:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.0.0
    container_name: 'elastic-search'
    ports:
      - '9200:9200'
      - '9300:9300'
    expose:
      - 9200
      - 9300
    environment:
      - discovery.type=single-node

As such, it might be worth fixing, or at least finding that config option and adding it to the docs for elastichq as a recommended setup step

[royrusso]

Tested with all versions and the fix should be working. I hope to do a release this week, if you don't want to pull from dev.

[swimmadude66]

Thanks!

[bliscprojects]

What is the solution to this problem? I am using Ubuntu 18.04

[stolario]

works this way:

{ "settings": { "number_of_shards": 100, "number_of_replicas": 2 }, "mappings": { ......