Indexer-less Graylog startup

[todvora]

What?

The graylog server should be able to start without any indexer instance (Elasticsearch/Opensearch/Datanode) and provide minimal interface to guide administrator through the indexer setup.

Why?

The current process requires all the configuration in place and correct, all dependencies (mongodb, indexer) available. There is no way to help users, suggest configuration for the dependencies, allow remote registration of indexer.

With a minimal boot interface (REST, web?), we'll be able to generate configuration files, certificates, accept certificate signing requests and generally make the initial setup easier.

[damianharouff]

This would be impactful for Cloud deployments too, where we need a runner to execute API tasks against the cluster if the main nodes of the cluster are not available.

[todvora]

Let's specify a bit what we are trying to achieve here and what steps are necessary to consider the feature complete.

• Graylog server provides a web interface that's available during the initial startup - even before the indexer (ES/OS/Datanode) is reachable.
• A limited set of configuration screens (webpages) exists, helping with indexer setup, supports embedded Graylog CA, possibly supports cloud setup or even whole graylog setup(generating graylog.conf)?
• Mongodb is available in this moment, but empty or mostly empty (fresh installation). Services may read and store values in mongodb (things like CSR for the CA)
• This initial setup interface terminates automatically or manually and the standard boot procedure continues.
• Securing the interface - only admin username/password from the graylog.conf available at the moment. So basic auth + request filter?
• Frontend probably totally disconnected from the rest of the app - separated bundle with just a handful of simple config pages?
• Backend reuses the jersey service and preflight modules. The preflight checks are what starts and keeps the backend running.
• Backend has to operate with very limited set of resources and services, that will be explicitly linked to the preflight web module.