search

Graylog2 / collector-sidecar

Manage log collectors through Graylog
https://www.graylog.org/
Other
268 stars 56 forks source link

support multiple configurations per collector #441

Closed mpfz0r closed 2 years ago

mpfz0r commented 2 years ago

This PR contains 3 changes.

1. Refactor the sidecar so it can handle more than one configuration per backend.

The sidecar has three different global "stores" where it keeps the current state:

All of these stores are currently tied together with the collector id. If we want to support multiple instances of one collector, we need to use a unique ID for this. The obvious choice is to build an ID out of the collector and the configuration ID. (collector_id-configuration_id)

An assignment like:

     [
       { "collector_id": "1234", "configuration_id": "aaa" },
       { "collector_id": "1234", "configuration_id": "bbb" },
       { "collector_id": "5678", "configuration_id": "eee" }
     ]

Will create 3 backends: 1234-aaa, 1234-bbb, 5678-eee

2. Report the collector_configuration_directory back to Graylog

This allows Graylog to create a sidecar config variable ${sidecar.spoolDir} would expand to sth like /var/lib/graylog-sidecar/generated/62864e0056caeb4862bf016b/

To make this backwards compatible with older Graylog servers that won't accept additional NodeDetail fields in the API. (:roll_eyes:) we need to add a version check, and only send this to servers that support it.

3. Backwards compatible collector status.

The status of each collector was reported back to Graylog by using the collector_id. Additionally, we will now report this with the assigned configuration_id. But only to Graylog > 4.3

Refs https://github.com/Graylog2/graylog-plugin-enterprise/issues/3956