Evebox with ELK + X-Pack enabled

[ManuelFFF]

Hi,

I am currently running SELKS 6 (which includes ELK 7.8.1 + Suricata + Evebox). Everything was working fine until I enabled X-Pack on ELK with the basic security features. Now in order to connect to Elasticsearch you need to provide username and password. Currently Evebox is failing to connect to Elasticsearch:

Aug 12 10:09:29 server1 evebox[645]: 2020-08-12 10:09:29 (server.go:335) -- Failed to ping Elastic Search, delaying startup: : Get "http://localhost:9200/": dia…nection refused Aug 12 10:09:32 server1 evebox[645]: 2020-08-12 10:09:32 (server.go:335) -- Failed to ping Elastic Search, delaying startup: {"error":{"root_cause":[{"type":"security_exceptio…

How can I configure Evebox to use credentials to connect to Elasticsearch?

Thank you

[jasonish]

I think it should be enough to create a /etc/evebox/evebox.yaml that looks like:

database:
  elasticsearch:
    username: username
    password: password

[ManuelFFF]

Hi @jasonish ,

I appreciate your help. I'll be trying your solution shortly. Can you tell which permissions would need the user that Evebox will be using to connect to Elasticsearch? I plan to add a new user in ELK and assign some roll privileges depending on your advice.

Thank you in advance

[jasonish]

Within SELKS it will need read, and I think create as archiving, commenting or escalating an event needs to update the document. If create isn't enough, it might need write.

[ManuelFFF]

Hi @jasonish ,

Please forgive mi ignorance, as I don't have too much experience with ELK. Would it be too much ask you to please take a look at the Security Privileges page, and tell me the list of cluster and index privileges that user evebox_writer will need to connect to Elasticsearch and function correctly?

I not sure if this user will need cluster "monitor" privileges, or any other or none. I found at least three different index privileges and I am not sure which one I should select: "create", "create_doc", "create_index", "write" and also found "read" and might consider "view_index_metadata".

Thank you in advance

[jasonish]

I actually haven't used X-Pack myself, so I'm just going from the information here: https://www.elastic.co/guide/en/elasticsearch/reference/current/security-privileges.html

From here the permissions that stand out to me are:

• create (index privilege)
• read

I think thats it. Failing that you may have to do some trial and error. Assuming you created an evebox.yaml like mentioned above, you could run EveBox on the command line to get a better idea of the error's involved and report back:

evebox server -c /etc/evebox/evebox.yaml -p 9999

the connect port 9999. You might also get the same info out of journalctl -xf | grep evebox as well though.

[ManuelFFF]

Hi @jasonish ,

I created the role and user for Evebox in Elasticsearch. Added the /etc/evebox/evebox.yaml and restarted evebox service, but still cannot access Elasticsearch. Just for testing I am using the "elastic" user (superadmin with all privileges). After checking the journal, it looks like Evebox is using a configuration database file. I believe the settings I need to change/add (without breaking SELKS) lies in this file. Can you help me to edit it?

journal output

user1@server1:~$ sudo systemctl restart evebox
user1@server1:~$ sudo journalctl -f -u evebox
-- Logs begin at Mon 2020-08-10 21:58:46 EDT. --
Aug 14 10:31:37 server1 evebox[26534]: 2020-08-14 10:31:37 (server.go:163) <Info> -- This is EveBox Server version 0.11.1 (rev: 4d1b355); os=linux, arch=amd64
Aug 14 10:31:37 server1 evebox[26534]: 2020-08-14 10:31:37 (server.go:252) <Info> -- Self test: found embedded index.html.
Aug 14 10:31:37 server1 evebox[26534]: 2020-08-14 10:31:37 (geoip-service.go:44) <Warning> -- Failed to initialize geoip database: no database files found
Aug 14 10:31:37 server1 evebox[26534]: 2020-08-14 10:31:37 (configdb.go:59) <Info> -- Using configuration database file /var/lib/evebox/config.sqlite
Aug 14 10:31:37 server1 evebox[26534]: 2020-08-14 10:31:37 (server.go:305) <Info> -- Configuring ElasticSearch datastore
Aug 14 10:31:37 server1 evebox[26534]: 2020-08-14 10:31:37 (server.go:306) <Info> -- Using ElasticSearch URL http://localhost:9200
Aug 14 10:31:37 server1 evebox[26534]: 2020-08-14 10:31:37 (server.go:308) <Info> -- Using ElasticSearch Index logstash.
Aug 14 10:31:37 server1 evebox[26534]: 2020-08-14 10:31:37 (elasticsearch.go:109) <Info> -- Event base index: logstash
Aug 14 10:31:37 server1 evebox[26534]: 2020-08-14 10:31:37 (elasticsearch.go:110) <Info> -- Event search index: logstash-*
Aug 14 10:31:37 server1 evebox[26534]: 2020-08-14 10:31:37 (server.go:335) <Error> -- Failed to ping Elastic Search, delaying startup: {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}},"status":401}
Aug 14 10:31:40 server1 evebox[26534]: 2020-08-14 10:31:40 (server.go:335) <Error> -- Failed to ping Elastic Search, delaying startup: {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}},"status":401}
Aug 14 10:31:43 server1 evebox[26534]: 2020-08-14 10:31:43 (server.go:335) <Error> -- Failed to ping Elastic Search, delaying startup: {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}},"status":401}

[jasonish]

I don't think its loading the config file by default like I thought it would. On SELKS, edit the file /etc/default/evebox and then uncomment the line that looks like:

#CONFIG="-c /etc/evebox/evebox.yaml

[ManuelFFF]

Hi @jasonish ,

By doing this I am forcing Evebox to use the config file /etc/evebox/evebox.yaml instead of /var/lib/evebox/config.sqlite. After restart evebox service it looks like is connecting to Elasticsearch as expected, however I am concerned if I am not missing any critical or important settings involved in SELKS internal workflow, as I don't know what other configs they have in that file that I am ignoring now. Do you know anything about it?

Thank you

journal output

user1@server1:~$ sudo journalctl -f -u evebox
[sudo] password for user1:
-- Logs begin at Mon 2020-08-10 21:58:46 EDT. --
Aug 14 16:06:22 server1 evebox[11721]: 2020-08-14 16:06:22 (server.go:305) <Info> -- Configuring ElasticSearch datastore
Aug 14 16:06:22 server1 evebox[11721]: 2020-08-14 16:06:22 (server.go:306) <Info> -- Using ElasticSearch URL http://localhost:9200
Aug 14 16:06:22 server1 evebox[11721]: 2020-08-14 16:06:22 (server.go:308) <Info> -- Using ElasticSearch Index logstash.
Aug 14 16:06:22 server1 evebox[11721]: 2020-08-14 16:06:22 (elasticsearch.go:109) <Info> -- Event base index: logstash
Aug 14 16:06:22 server1 evebox[11721]: 2020-08-14 16:06:22 (elasticsearch.go:110) <Info> -- Event search index: logstash-*
Aug 14 16:06:22 server1 evebox[11721]: 2020-08-14 16:06:22 (server.go:338) <Info> -- Connected to Elastic Search (version: 7.8.1)
Aug 14 16:06:22 server1 evebox[11721]: 2020-08-14 16:06:22 (elasticsearch.go:177) <Info> -- Assuming Logstash style index
Aug 14 16:06:22 server1 evebox[11721]: 2020-08-14 16:06:22 (server.go:131) <Info> -- Session reaper started
Aug 14 16:06:22 server1 evebox[11721]: 2020-08-14 16:06:22 (server.go:165) <Info> -- Authentication disabled.
Aug 14 16:06:22 server1 evebox[11721]: 2020-08-14 16:06:22 (server.go:261) <Info> -- Listening on [127.0.0.1]:5636

[jasonish]

The config file and the config database hold different information. So don't worry about that.

I believe SELKS 6 is still using Logstash to add events to Elasticsearch. So you'll probably have to add something to that configuration as well. I'm not much help to you there though.

[ManuelFFF]

Hi @jasonish ,

You have been incredible helpful and fast supporter. I can't tell you how grateful I am right now. I was able to resolve the issue with your help in a very short time. And regarding Logstash you are right. Logstash will need credentials as well, but I already took care of that and it is connecting to Elasticsearch without issues, but I appreciate that you mentioned.

I wish you an excellent weekend! Thank you

[ManuelFFF]

@jasonish ,

I am sorry, but I forgot to ask before. Within the evebox.yaml, Elasticsearch credentials are exposed as plain text. Is there a way to change that? For instance Kibana and Logstash can use a keystore to store credentials securely. If Evebox does not have such a thing like keystore file, then having those credentials added to the config.sqlite file will serve as a workaround?

Thank you

[jasonish]

config.sqlite won't help as its essentially plain text. The only way you can really do this with EveBox is to make the permissions on evebox.yaml are tight. I took a look at the Logstash keystore configuration, and its not that much secure given they suggest dropping the keystore password as an environment variable and securing the permissions on that file.

At most I'd probably let the credentials be specified as environment variables. I'm not sure how well that would work in a scenario like SELKS, but would help with secrets databases like Vault, etc.

[ManuelFFF]

@jasonish ,

Thank you a lot for all your help!