openshift / origin-aggregated-logging

139 stars 230 forks source link

Do not set mapper.allow_dots_in_name=true for ES 5.x and later #1666

Closed lukas-vlcek closed 4 years ago

lukas-vlcek commented 5 years ago

I removed this settings in #1188 but it does not seem to be enough. The following is ES node log in OCP 4.1 (see the ES_JAVA_OPTS variable):

[2019-06-12 11:16:54,209][INFO ][container.run            ] Begin Elasticsearch startup script
[2019-06-12 11:16:54,213][INFO ][container.run            ] Comparing the specified RAM to the maximum recommended for Elasticsearch...
[2019-06-12 11:16:54,216][INFO ][container.run            ] Inspecting the maximum RAM available...
[2019-06-12 11:16:54,218][WARN ][container.run            ] Setting the maximum allowable RAM to 600m which is the largest amount available
[2019-06-12 11:16:54,219][INFO ][container.run            ] ES_JAVA_OPTS: '-Dmapper.allow_dots_in_name=true -Xms300m -Xmx300m'
[2019-06-12 11:16:54,221][INFO ][container.run            ] Copying certs from /etc/openshift/elasticsearch/secret to /etc/elasticsearch/secret
[2019-06-12 11:16:54,225][INFO ][container.run            ] Building required jks files and truststore

We need to find out where this is being added into ES_JAVA_OPTS and remove it.

richm commented 5 years ago

Do we need to do this for ES6?

lukas-vlcek commented 5 years ago

This setting was introduced in ES 2.4 but it is no longer used in ES 5.x and later. I believe it was ignoder by ES 5.x but I would rather remove it. First, it does not apply, second, I can imagine Elasticsearch can become very strict and complain about this invalid setting on startup in later versions.

jcantrill commented 4 years ago

@lukas-vlcek did you find the source? do you have a PR to resolve? what is the impact?

lukas-vlcek commented 4 years ago

To me it seems it is part of the image.

$ oc rsh elasticsearch-cdm-mm11mrt6-1-fcfc79457-52bnk
Defaulting container name to elasticsearch.
Use 'oc describe pod/elasticsearch-cdm-mm11mrt6-1-fcfc79457-52bnk -n openshift-logging' to see all of the containers in this pod.

sh-4.2$ echo $ES_JAVA_OPTS
-Dmapper.allow_dots_in_name=true
----- ^^^^^^^^^^ -----

sh-4.2$ 
lukas-vlcek commented 4 years ago

OK, I found it here: https://access.redhat.com/containers/?tab=tech-details#/registry.access.redhat.com/openshift4/ose-logging-elasticsearch5 See the Env part that contains

ES_JAVA_OPTS=-Dmapper.allow_dots_in_name=true

Now, I need to track it back to its source :-)

lukas-vlcek commented 4 years ago

Opened a BZ https://bugzilla.redhat.com/show_bug.cgi?id=1789321

lukas-vlcek commented 4 years ago

Closing ticket. The issue is actually not in the code of this repo, it is in different repo, see the BZ link above.