graylog3 - default uses incompatible elasticsearch version

[sjwk]

According to the documentation, Graylog 3 is only compatible with Elasticsearch 6.x. But the example playbooks given, and the default variables in the role don't pass a version to the elasticsearch role, which by default will install the latest version (currently 7.1.1), which will presumably cause issues with Graylog.

The playbook/role documentation should contain a note that setting es_version is required and/or default variable set.

[mariussturm]

On what OS did you face this issue? At least on Debian/Ubuntu there is pin for ES 6.x https://github.com/Graylog2/graylog-ansible-role/blob/master/defaults/main.yml#L16

[sjwk]

Odd. It was on Ubuntu 16.04. Without adding the es_version to my host variables, it added in the repo for 7.x, and installed elasticsearch 7.1.1. I hadn't noticed at first, but some bits weren't working properly and saw I was getting incompatible version error messages in the logs. I removed and purged elasticsearch and then did an autoremove, and removed the repo files from /etc/apt and tried again, and it again created the repo for 7.x (aborted before it installed it).

I will try again on a clean VM when I have 5 minutes. This particular VM used to have ELK stack on it, but I did delete and purge all config and manually removed any left over config files and repo list definitions before starting with graylog.

[jalogisch]

This issue is fairly old and there hasn't been much activity on it. Closing, but please re-open if it still occurs.

[mjkelly]

Hi,

I saw this on a fresh install today on ubuntu 18.04. I see the elasticsearch role sets a version in defaults/main.yml. I got things working by explicitly specifying the latest ES 6.8.x version, like this:

$ head -n2  elastic.elasticsearch/defaults/main.yml 
---
es_version: "6.8.6"

Here's the definition in their github repo: https://github.com/elastic/ansible-elasticsearch/blob/master/defaults/main.yml#L2

Based on reading https://github.com/elastic/ansible-elasticsearch/blob/master/tasks/elasticsearch-Debian.yml#L87, they're going to install {{ es_package_name }}={{ es_version }}, which I'm guessing overrides the version pinning since there's no way to satisfy it.

It looks like unsetting es_version may work.

Version info

Here are the versions I installed via ansible-galaxy:

$ ansible-galaxy list
- graylog2.graylog-ansible-role, 3.0.1
- jdauphant.nginx, v2.21.2
- elastic.elasticsearch, 7.6.0

I'm running on Ubuntu 18.04:

mkelly@graylog2:~$ cat /etc/issue
Ubuntu 18.04.4 LTS \n \l

Let me know if you need any more info or want me to experiment on anything. Thanks!

[mjkelly]

Actually, while the logic right around https://github.com/elastic/ansible-elasticsearch/blob/master/tasks/elasticsearch-Debian.yml#L87 handles an empty es_package_name, I get an error if it's an empty string:

TASK [elastic.elasticsearch : set fact java_state to present] ********************************************
fatal: [graylog3.home.michaelkelly.org]: FAILED! => {"msg": "The conditional check 'es_java_install' faile
d. The error was: Version comparison: LooseVersion instance has no attribute 'version'\n\nThe error appear
s to have been in '/home/mkelly/.ansible/roles/elastic.elasticsearch/tasks/java.yml': line 3, column 3, bu
t may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be
:\n\n\n- name: set fact java_state to present\n  ^ here\n"}      

I also tried:

1. unsetting es_version -- this errors out early
2. setting es_version: "6.8" (no point release) -- this fails when installing the package:

TASK [elastic.elasticsearch : Debian - Ensure elasticsearch is installed] ********************************
fatal: [graylog3.home.michaelkelly.org]: FAILED! => {"cache_update_time": 1581962522, "cache_updated": false, "changed": false, "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\"     install 'elasticsearch-oss=6.8'' failed: E: Version '6.8' for 'elasticsearch-oss' was not found\n", "rc": 100, "stderr": "E: Version '6.8' for 'elasticsearch-oss' was not found\n", "stderr_lines": ["E: Version '6.8' for 'elasticsearch-oss' was not found"], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information..."]}

So, in my setup, setting es_version: "6.8.6" was the only thing that worked.

I hope this is useful info. :)

[ForestFeather]

Just wanted to add, on Ubuntu 20.04, I get the same issues as mjkelly. Setting es-version to "6.8.12" didn't work; I had to revert to 18.04. Apparently elasticsearch didn't like 20.04 due to missing /etc/securetty or something similar, and it wouldn't start with root user errors.

[malcyon]

I have updated the example in our README to include the es_version declaration. I also added a Molecule test that installs Graylog 3.3 and Elasticsearch 6.8.10. This example playbook works for me (it runs Ubuntu 20.04).

I notice that the Elasticsearch role's README does specify that es_version needs to be in a X.Y.Z format.

I am not 100% certain that the pinning of the ES os package @mariussturm mentions does anything at all. It certainly doesn't stop the installing of ES 7.x. I feel like maybe it should be taken out, but I am afraid to for fear that I missed something.

In any case, I think this is mostly a documentation issue. You need to be specifying es_version, and you need to be sure the version of Graylog you are installing works with it. I have tried to make this clear in the README.

It's a bit of an old issue, so I'm just going to close it out. But feel free to comment further if there are still issues.