Closed mmguero closed 10 months ago
this would be very useful, thank you.
I've got this mostly working in my development fork. At this point it's a lot of heavy testing and writing documentation, but it should work in the next release.
things I still need to figure out:
Alerts - Tags
visualization looks really weirdthings I'm going to live with:
This all seems to be working. I'll release for v23.10.0 and then open individual issues if they crop up later.
We've had a number of interested parties wanting to be able to store logs in both OpenSearch and ElasticSearch.
The index store that is managed by Malcolm will always (for the foreseeable future) be OpenSearch. This is due 1) to the licensing changes that initially caused us to go this direction and 2) to the features in Elasticsearch that would only be available to "premium" subscribers that we can get in open-source OpenSearch.
However, because we already support opensearch instances that are "local" vs. "remote" the code is already in place in Malcolm to use another data store into which to index the data. As of Q3 2023, the APIs between the two projects are still pretty similar.
So what I'm envisioning is this:
Should they choose Elasticsearch, Malcolm won't run its opensearch container (the same way it doesn't today if they choose "OpenSearch (remote)") and it also won't run the dashboards container (since OpenSearch Dashboards isn't going to run against ElasticSearch).
Let's be clear: malcolm won't run an instance of ElasticSearch or Kibana. But it will be able to index the network data into an ElasticSearch instance and the user would be able to see that data in their own Kibana.
Here are a few areas we'd need to look at:
We shouldn't need to worry about:
Honestly it's not too much that deals directly with the opensearch API. It's pretty easy to determine where those places are: since we're already handling remote opensearch instances, just look anywhere in the docker-compose file where we're referencing
.opensearch.primary.curlrc
and that is a place that is talking to OpenSearch, and as such a place we'd need to support ElasticSearch.