Shard value not being honored on creation of first index with aliases enabled.

jaegertracing / jaeger

CNCF Jaeger, a Distributed Tracing Platform

https://www.jaegertracing.io/

Apache License 2.0

20.6k stars 2.45k forks source link

Shard value not being honored on creation of first index with aliases enabled. #2289

Closed hgrant-ebsco closed 5 months ago

hgrant-ebsco commented 4 years ago

We are currently experimenting with enabling the roll over api via Jaeger. In doing so, we are noticing that even though we specify a shards value of 2, the first index is always created with 5 shards when we have aliases enabled. Once the second index for the rollover is created, it honors the value of 2 shards, and does so going forward.

If we do not enable alias, the shard value is honored from the start with the daily created indices.

Our Jaeger collector parameters are as follows:

` name: jaeger-collector args:

'--config-file=/conf/collector.yaml'
'--es.num-shards=2'
'--es.use-aliases=true'
'--es.num-replicas=1'
'--collector.queue-size=105000'
'--collector.num-workers=200'
'--log-level=warn'`

pavolloffay commented 4 years ago

@JayG-EBSCO did you run the init step with rollover job?

hgrant-ebsco commented 4 years ago

@JayG-EBSCO did you run the init step with rollover job?

@pavolloffay Yes, first we run a single run job using the es-rollover container to enable the aliases. Yaml for that as follows:

`--- apiVersion: batch/v1 kind: Job metadata: name: jaeger-enable-es-rollover labels: app: jaeger-enable-es-rollover spec: template: spec: containers:

name: jaeger-enable-es-rollover image: jaeger-es-rollover:1.18.0 args: ["init", "https://_ES_URL_"] env:
- name: TIMEOUT value: "30" restartPolicy: Never`

pavolloffay commented 4 years ago

thanks for the confirmation.

I am not sure if query can submit index templates. Is also query configured with the 2 shards?

hgrant-ebsco commented 4 years ago

thanks for the confirmation.

I am not sure if query can submit index templates. Is also query configured with the 2 shards?

Query currently does not have a shards value being passed to it. It's args are:

    args:
      - '--config-file=/conf/query.yaml'
      - '--es.use-aliases=true'
      - '--log-level=warn'

pavolloffay commented 4 years ago

I was not able to reproduce your issue:

docker run -it --rm -e "ES_JAVA_OPTS=-Xms2g -Xmx2g" -p 9200:9200 -p 9300:9300 -e "http.host=0.0.0.0" -e "discovery.type=single-node" --name=elasticsearch docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.4
docker run -it --rm --net=host -e SHARDS=1 -e REPLICAS=0 jaegertracing/jaeger-es-rollover:1.18.0 init http://localhost:9200
docker run -it --rm --net=host -e SPAN_STORAGE_TYPE=elasticsearch jaegertracing/all-in-one:1.18.0 --es.use-aliases=true

curl -ivX GET localhost:9200/jaeger-span-000001?pretty shows shards 1 and replicas 0

How are you deploying Jaeger? Via the Jaeger operator or your custom k8s manifests?

hgrant-ebsco commented 4 years ago

I was not able to reproduce your issue:
docker run -it --rm -e "ES_JAVA_OPTS=-Xms2g -Xmx2g" -p 9200:9200 -p 9300:9300 -e "http.host=0.0.0.0" -e "discovery.type=single-node" --name=elasticsearch docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.4
docker run -it --rm --net=host -e SHARDS=1 -e REPLICAS=0 jaegertracing/jaeger-es-rollover:1.18.0 init http://localhost:9200
docker run -it --rm --net=host -e SPAN_STORAGE_TYPE=elasticsearch jaegertracing/all-in-one:1.18.0 --es.use-aliases=true
curl -ivX GET localhost:9200/jaeger-span-000001?pretty shows shards 1 and replicas 0

How are you deploying Jaeger? Via the Jaeger operator or your custom k8s manifests?

We are currently using custom k8s manifests.

jkowall commented 5 months ago

Stale issue was never replicated, closing this out. Feel free to reopen.