opensearch-project / logstash-output-opensearch

A Logstash plugin that sends event data to a OpenSearch clusters and stores as an index.
https://opensearch.org/docs/latest/clients/logstash/index/
Apache License 2.0
104 stars 80 forks source link

[BUG] Exiting: error loading template: failed to load template: couldn't load template: 400 Bad Request #227

Open ZainHaiderArif opened 1 year ago

ZainHaiderArif commented 1 year ago

I am getting following erro while i am trying to configure filebeat with Opensearch and opensearch dashboard. when i try to configure filebeat with latest ELK docker image ir works fine but when i use opensearch 2.9.0 and opensearch dashboard while i setup file beat it is giving me this error Also i have attached log file

INFO template/load.go:132 Try loading template filebeat-7.15.0 to Elasticsearch 2023-09-05T18:14:55.175+0500 ERROR instance/beat.go:989 Exiting: error loading template: failed to load template: couldn't load template: 400 Bad Request filebeat.txt

dblock commented 1 year ago

Is there any more errors with that 400? Maybe on the server? Either way, describe your setup for beats, maybe a step-by-step repro from scratch?

dlvenable commented 1 year ago

@ZainHaiderArif , Can you provide your Logstash config file?

Also, I see the response from the server looks somewhat like the following:

"type":"mapper_parsing_exception","reason":"Root mapping definition has unsupported parameters:
ZainHaiderArif commented 1 year ago

Sorry for late reply Lemme explain a little i am using opensearch and opensearch dashoard This is my Opensearch running at 9201 { "name" : "opensearch-node1", "cluster_name" : "opensearch-cluster", "cluster_uuid" : "1jnF14_1TN-dq3EyhuKqDg", "version" : { "distribution" : "opensearch", "number" : "2.8.0", "build_type" : "tar", "build_hash" : "db90a415ff2fd428b4f7b3f800a51dc229287cb4", "build_date" : "2023-06-03T06:24:25.112415503Z", "build_snapshot" : false, "lucene_version" : "9.6.0", "minimum_wire_compatibility_version" : "7.10.0", "minimum_index_compatibility_version" : "7.0.0" }, "tagline" : "The OpenSearch Project: https://opensearch.org/" }

==================

Also I am using Opensearch dashboard docker image with that Image for opensearch : image: opensearchproject/opensearch:latest Image for OpensearchDashboard image: opensearchproject/opensearch-dashboards:latest

===========================

I am trying to send IIS metrices and logs using MetricBeat/FileBeat/HeartBeat By simply editing their yml file and setting output to 9201 elastic search

But then i get error related to template Exiting: error loading template: failed to load template: couldn't load

Also when i Test metric beat test output i get this D:\metricbeat\metricbeat-7.15.0-windows-x86_64>.\metricbeat test config Config OK -====================================================== D:\metricbeat\metricbeat-7.15.0-windows-x86_64>.\metricbeat test output elasticsearch: http://localhost:9200... parse url... OK connection... parse host... OK dns lookup... OK addresses: ::1, 127.0.0.1 dial up... OK TLS... WARN secure connection disabled talk to server... ERROR Connection marked as failed because the onConnect callback failed: could not connect to a compatible version of Elasticsearch: 400 Bad Request: {"error":{"root_cause":[{"type":"invalid_index_name_exception","reason":"Invalid index name [license], must not start with ''.","index":"_license","index_uuid":"na"}],"type":"invalid_index_name_exception","reason":"Invalid index name [license], must not start with ''.","index":"_license","index_uuid":"na"},"status":400}

D:\metricbeat\metricbeat-7.15.0-windows-x86_64>

@dlvenable I am not using Logastash i am using metricbaet/filebeat/heartbeat and changing their config files

========================== @dblock I installed 7.0.0 version of metricbeat from https://www.elastic.co/downloads/past-releases/metricbeat-7-0-0

Downloaded zip file enable service using .\metricbeat.exe enable modules iis and system then when i run ".\metricbeat.exe setup -e" i get this tempate error

When i trying same thing with Elastic/kibana latest image it works fine but having this issue with opensearch