Multi-target support

[HryhoriiDidenko]

Hey, from my perspective we need to move a elastic connection string configuration from a Helm release parameters configuration to the separate CDR. For example, this CDR should have a next name Cluster. After that, we can use the following approach, all CDRs related to elastic object creation need to include a new option with the name a Cluster configuration.

[xco-sk]

@HryhoriiDidenko thanks for the idea, it will help with separating the operator and target ES/Kibana configuration. I will try it out and see if it's feasible, but so far, I don't see any issue.

[HryhoriiDidenko]

@xco-sk Hey, think we need to stay available for connection string configuration from Helm release parameters, as default. It is needed for backward compatibility. But if we want to use a different cluster we need to create a new CDR with a connection string and point this CDR to specific resources.

[HryhoriiDidenko]

@xco-sk Do you have some README on how to start developing your service?

[xco-sk]

Hi @HryhoriiDidenko, sorry, no README about developing yet, I should create one. But essentially, all resources are scaffolded with kubebuilder. I will write at least a few points on how to develop new CRDs and what/where to implement the logic behind them.

[xco-sk]

@HryhoriiDidenko so I generated 2 new CRDs - KibanaInstance and ElasticsearchInstance using kubebuilder commands:

kubebuilder create api --group es.eck --version v1alpha1 --kind ElasticsearchInstance
kubebuilder create api --group kibana.eck --version v1alpha1 --kind KibanaInstance

During the generate, I set the "Create controller" to "no" - we should not need controllers for these as they just hold the data and we don't need to monitor their changes.

Then there are some manual steps required, see my feature branch commit where I outlined what's necessary in order to implement the multi-target feature with the usage of newly generated CRDs.

[georambo]

any idea when this might be tested and merged in? it would be a very useful feature. i have mutliple elasticsearch clusters, and it would be great to be able use these custom resources for all of them.

[xco-sk]

Hi @georambo, the implementation is in progress. Sorry, it takes so long, I underestimated the number of changes needed (e.g. the resource delete in ES controllers has to be refactored to the "finalizers" style, same as we got in Kibana controllers). Will keep you posted here, thanks for the patience :-)

[georambo]

No worries, I appreciate the effort on this operator, and it's very useful.

[xco-sk]

I'm finalizing the implementation and testing of the feature, so in the next couple of days, the version with multi-target support will be released if no major issue appears.

[xco-sk]

The multi-target support is now implemented in v0.5.0, if you use helm, please run helm repo update beforehand to get the latest helm chart. The migration guide is present in the README. There is also detailed documentation of 2 new CRDs present, see ElasticsearchInstance and KibanaInstance. See also updated documentation of other resources with a description of new spec.targetInstance.name field.

The change is backward compatible - when the field is not present, the default configuration (operator config( is being used.

Please let me know if there is an issue or if help is needed.

[georambo]

works like a charm. great stuff. thank you.