Need to create all indices using only one unassigned shard

[sharmin-anee]

Hi, I'm back again.

This time I'm facing another problem. The python script creates one unassigned shard per index. Is this possible or not to keep the unassigned shard to 1? If possible, then how can I do this?

Or how can I reduce the total no. of unassigned shards?

Thanks.

[roiravhon]

Im guessing that the unassigned shard you are talking about is because of the replica. Elasticsearch won't hold both the primary shard and the replica on the same node, and in case you don't have a cluster setup, it would be unassigned.

Try to set --number-of-replicas to 0, then, Elasticsearch should be green.

[sharmin-anee]

I set --number-of-replicas to 0. but unassigned shards also creating. I've one master node and one client node. If I take 2 client nodes, then will it be okay?

[roiravhon]

How many data nodes you got? If you have 0 replicas that means the primaries cant be assigned.

So either no data nodes, or something prevent them from being allocated (full disk, some shard awareness, etc)

[sharmin-anee]

used 0 data nodes.

[roiravhon]

So.. You need at least one :)

[sharmin-anee]

python elasticsearch-stress-test.py --es_address ip_address --indices 10 --documents 10 --clients 1 --seconds 300 --number-of-shards 1 --number-of-replicas 0 --not-green --seconds 120 --no-cleanup

What will be the ip_address here, data node ip_address or client node ip_address?

[roiravhon]

All will work.

But client node will be the best option as that is its job.

[sharmin-anee]

I used client ip_address and it creates equal no. of shards & unassigned shards. Cluster status is yellow now.

What to do now?

[roiravhon]

Sure about the --number-of-replicas 0 flag?

Try to delete all indices and run again. If persists, send me the output of

curl ip:9200/_cat/indices

[sharmin-anee]

python elasticsearch-stress-test.py --es_address ip_address --indices 10 --documents 50 --number-of-shards 1 --number-of-replicas 0 --seconds 120 --not-green --clients 2 --no-cleanup

curl ip:9200/_cat/indices

green open vmhysxht 1 0 2054 0 55.8mb 55.8mb green open iifwy 1 0 1996 0 61.7mb 61.7mb green open vldyizta 1 0 2252 0 64mb 64mb green open v 1 0 1952 0 58.5mb 58.5mb green open w 1 0 2038 0 57.8mb 57.8mb green open fgevvzoyass 1 0 1940 0 60.1mb 60.1mb green open xhmvibdmbdnzj 1 0 2127 0 62.1mb 62.1mb green open mtlkwnqaxy 1 0 1977 0 59.1mb 59.1mb green open laoriqxlsrrl 1 0 1879 0 54.3mb 54.3mb green open qvokuxy 1 0 2027 0 57.3mb 57.3mb

[roiravhon]

LGTM.. You cluster should be green and the test looks OK now.

[sharmin-anee]

But is has 10 shards

cluster health status

epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent 1468150696 07:38:16 el-cluster-1 green 3 1 10 10 0 0 0 0 - 100.0%

[roiravhon]

You are running the test on 10 indices, with 1 shard and no replicas per index. How many are you expecting to see?

[sharmin-anee]

is it creating one shard per index? why so much shard I need?

[sharmin-anee]

when I used 100 indices then it creates equal no. of shards & unassigned shards.

python elasticsearch-stress-test.py --es_address ip_address --indices 100 --documents 500 --number-of-shards 0 --number-of-replicas 0 --seconds 120 --not-green --clients 2 --no-cleanup

epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent 1468150595 07:36:35 el-cluster-1 yellow 3 1 495 495 0 0 495 0 - 50.0%

[roiravhon]

That is how elasticsearch works. Take a look here: https://www.elastic.co/guide/en/elasticsearch/reference/current/_basic_concepts.html

[roiravhon]

You used number-of-shards 0 which is not acceptable.

Also, remove the --no-cleanup, or cleanup manually between tests, you are seeing indices from older tests