search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.35k stars 1.06k forks source link

Does the open source version of Graylog 6.0.5 support Elasticsearch 7.10.2? #20428

Open ysj2018 opened 2 weeks ago

ysj2018 commented 2 weeks ago

Expected Behavior

I want Graylog to function normally.

Current Behavior

After I start Graylog, I keep getting the following error(s) ：

2024-09-12 18:43:41,201 ERROR: org.graylog2.storage.versionprobe.VersionProbe - Unable to retrieve version from OpenSearch/Elasticsearch node 10.80.69.27:9200: unknown error - an exception occurred while deserializing error response: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `org.graylog2.storage.versionprobe.Error` (although at least one Creator exists): no String-argument constructor/factory method to deserialize from String value ('Not Found')
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 67] (through reference chain: org.graylog2.storage.versionprobe.ErrorResponse["error"])
2024-09-12 18:43:41,201 INFO : org.graylog2.storage.versionprobe.VersionProbe - OpenSearch/Elasticsearch is not available. Retry #1158
2024-09-12 18:43:46,203 ERROR: org.graylog2.storage.versionprobe.VersionProbe - Unable to retrieve version from OpenSearch/Elasticsearch node 10.80.69.27:9200: unknown error - an exception occurred while deserializing error response: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `org.graylog2.storage.versionprobe.Error` (although at least one Creator exists): no String-argument constructor/factory method to deserialize from String value ('Not Found')
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 67] (through reference chain: org.graylog2.storage.versionprobe.ErrorResponse["error"])
2024-09-12 18:43:46,203 INFO : org.graylog2.storage.versionprobe.VersionProbe - OpenSearch/Elasticsearch is not available. Retry #1159
2024-09-12 18:43:51,205 ERROR: org.graylog2.storage.versionprobe.VersionProbe - Unable to retrieve version from OpenSearch/Elasticsearch node 10.80.69.27:9200: unknown error - an exception occurred while deserializing error response: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `org.graylog2.storage.versionprobe.Error` (although at least one Creator exists): no String-argument constructor/factory method to deserialize from String value ('Not Found')
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 67] (through reference chain: org.graylog2.storage.versionprobe.ErrorResponse["error"])
2024-09-12 18:43:51,205 INFO : org.graylog2.storage.versionprobe.VersionProbe - OpenSearch/Elasticsearch is not available. Retry #1160

Possible Solution

Does the open source version of Graylog 6.0.5 support Elasticsearch 7.10.2?

Steps to Reproduce (for bugs)

1.I installed MongoDB 7.0.11 2.I installed elasticsdearch7.10.2 3.I have downloaded the tar package of Graylog 6.0.5, but after configuring and starting it, I keep getting the above-mentioned error. I was able to run Graylog Enterprise Edition normally before, but the open-source version is not working despite similar configurations. I'm not sure what the exact issue is. Could you please tell me if Graylog 6.0.5 open-source edition supports Elasticsearch 7.10.2? Is there anything I might have done wrong?"

Regarding the support for Elasticsearch 7.10.2 with Graylog 6.0.5 open-source edition, it's important to consult the official documentation or release notes of Graylog to verify the compatibility. It's possible that the open-source edition might have different support requirements or limitations compared to the enterprise edition

Context

Your Environment

dennisoelkers commented 2 weeks ago

What happens when you try to fetch 10.80.69.27:9200 with curl in the same way that you configured Graylog with (i.e. http/https, auth, ...)?

ysj2018 commented 2 weeks ago

My Elasticsearch is not encrypted, and I can access it normally and get results using curl. However, when I use Graylog, it keeps reporting errors. I've reinstalled it on this machine, but the issue persists. I would like to ask if you can confirm that the open-source version of Graylog supports Elasticsearch 7.10.2? If so, I will try reinstalling everything on a different machine to ensure there are no unknown issues with the source data.

T100D commented 2 weeks ago

What happens if you take a lower version of MongoDB 6 or even 5 if that is supported. Seen some problems mentioned in that corner to.

dennisoelkers commented 2 weeks ago

@ysj2018: We do use ES7 7.10.2 in automated tests (e.g. this one) which are bootstrapping a complete Graylog server + Elasticsearch instance. Therefore I can safely confirm that it supports ES7. Can you post the results of the curl call to http://10.80.69.27:9200/ from your Graylog node here, so I can verify that there is no parsing issue?

T100D commented 2 weeks ago

@dennisoelkers

Is the oss version of elasticsearch used, the normal version does not work out of the box with Graylog.

https://www.docker.elastic.co/r/elasticsearch/elasticsearch-oss

dennisoelkers commented 2 weeks ago

Hey @T100D,

we are using the same image in our automated tests: https://github.com/Graylog2/graylog2-server/blob/6.0.5/graylog-storage-elasticsearch7/src/test/java/org/graylog/storage/elasticsearch7/testing/ElasticsearchInstanceES7.java#L74

T100D commented 2 weeks ago

@ysj2018

Asked the wrong person........

Is the oss version for elasticsearch used, the normal version does not work out of the box with Graylog. https://www.docker.elastic.co/r/elasticsearch/elasticsearch-oss

Java version could be wrong to, Graylog uses it's own binaries of Java and I do not know if Java 17 is supported bij elasticsearch.

We use openjdk version "1.8.0_422"