jaegertracing / jaeger

CNCF Jaeger, a Distributed Tracing Platform
https://www.jaegertracing.io/
Apache License 2.0
20.46k stars 2.44k forks source link

More valid values for frequency of index rollover in Elasticsearch #3549

Open ADeeeee opened 2 years ago

ADeeeee commented 2 years ago

Requirement - what kind of business use case are you trying to solve?

Jaeger only support "Day and Hour" for the suffix of index in Elasticsearch.

e.g.

--es.index-rollover-frequency-services

Rotates jaeger-service indices over the given period. For example "day" creates "jaeger-service-yyyy-MM-dd" every day after UTC 12AM. Valid options: [hour, day].

Problem - what in Jaeger blocks you from solving the requirement?

I have only few data for debug in Jaeger every day, but it rollovers by date, which makes many indexes in my external ES which already exist and also used by other services while using Jaeger.

It shall need to delete the old indexes daily with the jaeger-es-rollover daemon, but I don't really know wether it'd delete my indexes from other projects or not.

Proposal - what do you suggest to solve the problem or improve the existing situation?

Support "Month, Year" for rollover in Jaeger.

The ideal index suffix should be jaeger-*-yyyy or jaeger-*-yyyy-MM

Any open questions to address

There seems no relative problems on the internet. Not really sure because the key words, e.g. "jaeger elasticsearch index suffix date, aren't that exactly correct

Is there any other way for workaround so far? Any information or help would be appreciated.

SSaMKJ commented 2 years ago

I need too.

Azim-Palmer commented 2 years ago

Yes please. We have daily logs sizes of <10megs but they were auto creating indexes, even after setting the primary shards down to 1 this is still incredibly excessive.

There's some experimental ILM support that complicates setup. Being able to set a single command line option is a more elegant solution for people with my low log entry use case which I suspect includes a lot of people starting with Jaeger.

mygit-2023 commented 1 year ago

I need too.

Ahsan-Aziz-Ishan commented 7 months ago

I need too.