[BUG] In case of multiple end-points defined in opensearch.hosts - the events are being distributed using a round-robin mechanism instead of being sent to all endpoints

[leonsery]

In case of multiple end-points defined in opensearch.hosts - the events are being distributed using a round-robin mechanism instead of being sent to all endpoints

To Reproduce

1. Define 2 end-points in logstash opensearch.hosts configuration, for example: opensearch { hosts => ["127.0.0.1:9200", "10.10.10.10:9200"]
2. Send several events via logstash
3. Some events appear in 1st instance, others - in 2nd instance

Expected behavior All events should appear in both opensearch instances.

Plugins

• input:

  1. beats

• filter

  1. grok
  2. json
  3. date
  4. ruby

• output

  1. opensearch

Host/Environment (please complete the following information):

• OS: Windows

[dlvenable]

@leonsery , Thank you for reaching out to us. The multiple hosts is how the output plugin is designed and this is inline with how the OpenSearch client works. You specify a list of OpenSearch hosts within your cluster.

If you wish to send all events to multiple clusters, you should define multiple output plugins

output {
  opensearch {
    hosts => ["127.0.0.1:9200"]
  }
  opensearch {
    hosts => ["10.10.10.10:9200"]
  }
}

This will send all events to both OpenSearch clusters.

I hope this helps. Let us know if this solves your problem or you have further questions.

[leonsery]

Thanks a lot! This was my misunderstanding approved by ChatGPT :-)

вт, 9 апр. 2024 г. в 22:40, David Venable @.***>:

@leonsery https://github.com/leonsery , Thank you for reaching out to us. The multiple hosts is how the output plugin is designed and this is inline with how the OpenSearch client works. You specify a list of OpenSearch hosts within your cluster.

If you wish to send all events to multiple clusters, you should define multiple output plugins

output { opensearch { hosts => ["127.0.0.1:9200"] } opensearch { hosts => ["10.10.10.10:9200"] } }

This will send all events to both OpenSearch clusters.

I hope this helps. Let us know if this solves your problem or you have further questions.

— Reply to this email directly, view it on GitHub https://github.com/opensearch-project/logstash-output-opensearch/issues/244#issuecomment-2045931003, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGPDKTWTYHFS3YD7BT2UB73Y4Q7ZJAVCNFSM6AAAAABF7DME2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBVHEZTCMBQGM . You are receiving this because you were mentioned.Message ID: <opensearch-project/logstash-output-opensearch/issues/244/2045931003@ github.com>

-- Regards, Leon Sery.