search

lmenezes / cerebro

MIT License
5.51k stars 714 forks source link

Chef Cookbook with TLS/SSL for Elasticsearch 6.2, Kibana, and Cerebro Available #293

Open cmcconnell1 opened 6 years ago

cmcconnell1 commented 6 years ago

Hello,

Thanks for your work on cerebro, we are now using it and wanted to give back. I was recently working on getting Elasticsearch 6.2, Kibana, and Cerebro automated/deployed with TLS/SSL certs, keys, keystore, etc. for our internal infrastructure. It's not perfect by any means, but works for us and I'm wondering if it might be helpful for others?:

6.x Elasticsearch Chef Cookbook with TLS/SSL, Kibana, and Cerbro for management, etc.

Regarding the cerebro project, I didn't see a 'contrib' dir, and not sure you'd want one?
Additionally, the cookbook that I wrote is large with a lot of stuff you probably don't need (elasticsearch clusters and kibana coordinating nodes, etc.), but might be useful in getting 'something' out there for automating cerebro install/config for the latest Elasticsearch 6.x with x-pack and the new TLS requirements, etc. It's a bit tricky to get it right.

Quick Summary: For cerebro, I based the automation on what I could find--this one was a gem: Access To Elasticsearch With Cerebro Via SSL+LDAP

Overview of the Kibana and Cerebro components of that cookbook

Kibana (HTTPS/SSL) "Coordinating Node"

Cerbro (HTTPS/SSL) on the Kibana node

Anyway, I am hoping that perhaps some of this work could be useful to other users of the cerebro project, and I could do a p/r with a ./contrib/chef dir and maybe some doc about automating it if desired. I haven't tried to decouple the cerebro recipe from the cookbook as it uses the code and files from them.

Thanks, -cmcc

abessifi commented 5 years ago

but might be useful in getting 'something' out there for automating cerebro install/config for the latest Elasticsearch 6.x with x-pack and the new TLS requirements, etc. It's a bit tricky to get it right.

@cmcconnell1 do you mean that Cerebro is not easy to install for ES 6.x out of the box ?

cmcconnell1 commented 5 years ago

Hey @abessifi (regarding my above comment five months ago) Since ES 6.x introduced required TLS encryption with keys etc., it was a pretty big change from the previous major versions that we were running which didn't use TLS and didn't have to have CA's to run ES queries, etc. And you can't license a 6.x cluster unless you have the certs installed/configured etc. AIR at that time, the ES docs were lacking many requisite details, etc. And regarding cerebro, at that time, I didn't find any updated automated install CM (chef, etc.) cookbooks/playbooks available online that showed how to handle the requisite certs, etc. TLS is also a requirement for 6.x and onward for kibana, etc.
So, at that time of my above comment (five months ago) I don't think anyone had any working examples showing the certs requirements for a lot of these ES-stack related components. I chose to stick our certs in locked-down S3 buckets (which for us are created for each ES cluster before deploying and are available to chef nodes via IAM policy and roles). Chef and databags is not a good idea and encrypted data bags are even worse. Regarding ES stack and certs, I wasted a lot of time trying to hack our automation for 6.x ES stack to use (CSR) certs that were signed by Digi, only to find out that those would never work with ES--if I recall correctly we couldn't use our wildcard certs since they were missing the hostname, etc. That was also not in the docs. Also, at that time you could only use PEM formatted certs (i.e. PKCS12 wouldn't work && found out the hard way there too). Of course, this may have changed. YMMV. ;-)