Closed ManuelFFF closed 4 years ago
There is no way right now.
Do you have a PEM or PKCS12 client certificate you have to present? Is it password protected?
Hi,
I have PKCS12 certificate without password.
Thank you
Ok, this is something I'll need to add.. Its somewhere on the todo list anyways. Will try to add in the next few days, and I hope you can try out a development build.
Thank you!
I have a development build here: https://evebox.org/files/development/evebox-latest-linux-x64.zip
On first look I thought you might have been trying to use client certificate based authentication, but realized that EveBox simply had an issue connecting to any server with a self-signed certificate, basically disable-certificate-check: true
was not working. This build should fix this.
You can just unzip the build and run ./evebox server -c /path/to/config
...
I'll try that today. Thank you!
I haven't had the opportunity to try this out, as I was struggling with TLS and Logstash all day, but this has been resolved, so I promise that tomorrow morning I will dedicate time to Evebox.
Thank you
Hi @jasonish ,
Finally I was able to fix Logstash and beats to access Elasticsearch with TLS enabled. Now I am testing Evebox. This is what I did:
/usr/bin/evebox
Remember that I am running SELKS 6, which was using the sqlite config file.
For the current config you recommended to modify file /etc/default/evebox
to use a yml
config file. With new dev Evebox, now I can access the site, but it does not looks like it's connecting to Elasticsearch and it keeps loading forever. I will share previous and current log output:
# The URL to Elastic Search. Setting it here will override the URL set
# in the config file if used.
#ELASTICSEARCH_URL="-e http://localhost:9200"
# Config file.
CONFIG="-c /etc/evebox/evebox.yaml"
# Other options.
EVEBOX_OPTS=""
database:
elasticsearch:
url: https://127.0.0.1:9200
username: elastic
password: [elastic_password]
disable-certificate-check: true
Aug 31 10:03:04 server1 evebox[6287]: 2020-08-31 10:03:04 (server.go:252) <Info> -- Self test: found embedded index.html.
Aug 31 10:03:04 server1 evebox[6287]: 2020-08-31 10:03:04 (geoip-service.go:44) <Warning> -- Failed to initialize geoip database: no database files found
Aug 31 10:03:04 server1 evebox[6287]: 2020-08-31 10:03:04 (configdb.go:59) <Info> -- Using configuration database file /var/lib/evebox/config.sqlite
Aug 31 10:03:04 server1 evebox[6287]: 2020-08-31 10:03:04 (server.go:305) <Info> -- Configuring ElasticSearch datastore
Aug 31 10:03:04 server1 evebox[6287]: 2020-08-31 10:03:04 (server.go:306) <Info> -- Using ElasticSearch URL https://127.0.0.1:9200
Aug 31 10:03:04 server1 evebox[6287]: 2020-08-31 10:03:04 (server.go:308) <Info> -- Using ElasticSearch Index logstash.
Aug 31 10:03:04 server1 evebox[6287]: 2020-08-31 10:03:04 (elasticsearch.go:109) <Info> -- Event base index: logstash
Aug 31 10:03:04 server1 evebox[6287]: 2020-08-31 10:03:04 (elasticsearch.go:110) <Info> -- Event search index: logstash-*
Aug 31 10:03:06 server1 evebox[6287]: 2020-08-31 10:03:06 (server.go:335) <Error> -- Failed to ping Elastic Search, delaying startup: : Get "https://127.0.0.1:9200/": x509: certificate
Aug 31 10:03:09 server1 evebox[6287]: 2020-08-31 10:03:09 (server.go:335) <Error> -- Failed to ping Elastic Search, delaying startup: : Get "https://127.0.0.1:9200/": x509: certificate
Aug 31 10:07:00 server1 systemd[1]: Started EveBox Server.
Aug 31 10:07:00 server1 evebox[6500]: 2020-08-31 10:07:00 INFO evebox::version: This is EveBox version 0.12.0-dev (rev: b94bae1); x86_64-unknown-linux-musl
Aug 31 10:07:00 server1 evebox[6500]: 2020-08-31 10:07:00 INFO evebox::server::main: Using temporary in-memory configuration database
Aug 31 10:07:00 server1 evebox[6500]: 2020-08-31 10:07:00 INFO evebox::sqlite::configrepo: Initializing SQLite database
Aug 31 10:07:00 server1 evebox[6500]: 2020-08-31 10:07:00 INFO evebox::sqlite::configrepo: Updating SQLite database to schema version 1
Aug 31 10:07:00 server1 evebox[6500]: 2020-08-31 10:07:00 ERROR evebox::server::main: Failed to get Elasticsearch version, things may not work right: error=request: error sending requ
Aug 31 10:07:00 server1 evebox[6500]: 2020-08-31 10:07:00 INFO evebox::server::main: Starting server on 127.0.0.1:5636, tls=false
Aug 31 10:08:23 server1 evebox[6500]: 2020-08-31 10:08:23 INFO evebox::server::main: Creating anonymous session for user from Some(V4(127.0.0.1:46162)) with name user1
Aug 31 10:08:23 server1 evebox[6500]: 2020-08-31 10:08:23 ERROR evebox::server::api::api: alert query failed: elastic search error
I don't have a work-around for you yet. But I see that I'm not logging enough data in that error to debug this, so I have a new build with better logging available at: https://evebox.org/files/development/evebox-latest-linux-x64.zip
I have tested this with my own install of Elasticsearch with authentication enabled and using a self-signed certificate. I also tested with Elastic's hosted cloud support, and it works.
Have you setup Elasticsearch to require client certification authentication?
Hi,
As always, I appreciate your prompt response. I have configured ELK to use a security certificate (self signed for now). So Elasticsearch is accepting only https connections that will also require a certificate. I will be testing the new dev version shortly and will provide the feedback.
Thank you
Ok, I don't have support yet for EveBox preventing a client certificate to Elasticsearch. It appears the hosted Elasticsearch service (by Elastic) doesn't allow this to be configured. Self hosted does, but I'm not yet familiar enough to configure that.
So for Logstast, Beats or Kibana, you had to install a client certificate before Elasticsearch would accept those connections?
Well, when I first installed SELKS 6, all apps were running and connecting without any issues. Then I enabled the X-Pack basic settings in Elasticsearch. First it was the basic authentication, with only username and password involved. All apps were able to connect to Elasticsearch. Finally I enabled the https access in Elasticsearch, and then all apps/client connecting to the node require a certificate. Kibana can use a PKCS12 certificate, while Logstash and the beats can read only PEM format. Without a certificate, the connection won't be allowed.
I tested the new dev. It looks like Evebox still can't connect to Elasticsearch.
user1@server1:~$ sudo systemctl start evebox
user1@server1:~$ sudo systemctl status evebox
ā evebox.service - EveBox Server
Loaded: loaded (/lib/systemd/system/evebox.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2020-08-31 15:49:06 EDT; 20s ago
Main PID: 25427 (evebox)
Tasks: 5 (limit: 4915)
Memory: 8.8M
CGroup: /system.slice/evebox.service
āā25427 /usr/bin/evebox server -c /etc/evebox/evebox.yaml
Aug 31 15:49:06 server1 systemd[1]: Started EveBox Server.
Aug 31 15:49:06 server1 evebox[25427]: 2020-08-31 15:49:06 INFO evebox::version: This is EveBox version 0.12.0-dev (rev: 3e56627); x86_64-unknown-linux-musl
Aug 31 15:49:06 server1 evebox[25427]: 2020-08-31 15:49:06 INFO evebox::server::main: Using temporary in-memory configuration database
Aug 31 15:49:06 server1 evebox[25427]: 2020-08-31 15:49:06 INFO evebox::sqlite::configrepo: Initializing SQLite database
Aug 31 15:49:06 server1 evebox[25427]: 2020-08-31 15:49:06 INFO evebox::sqlite::configrepo: Updating SQLite database to schema version 1
Aug 31 15:49:06 server1 evebox[25427]: 2020-08-31 15:49:06 ERROR evebox::server::main: Failed to get Elasticsearch version, things may not work right: error=request: error sending req
Aug 31 15:49:06 server1 evebox[25427]: 2020-08-31 15:49:06 INFO evebox::server::main: Starting server on 127.0.0.1:5636, tls=false
Please, keep me posted about any progress. I am available for testing and sharing feedback. Thank you
Does curl -Lv https://your-elastic:9200
work?
Or if not, curl -Lkv https://...
?
It seems to work
user1@server1:~$ curl -Lv https://192.168.1.17:9200
* Expire in 0 ms for 6 (transfer 0x5646400bea90)
* Trying 192.168.1.17...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5646400bea90)
* Connected to 192.168.1.17 (192.168.1.17) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: self signed certificate in certificate chain
* Closing connection 0
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
user1@server1:~$
No, that didn't work. You should see some JSON data telling you to authenticate. Add -k
and see if you get it the JSON error. Further, you could add -u username:password
to see if you actually get the Elasticsearch version.
user1@server1:~$ curl -Lv https://192.168.1.17:9200 -u elastic:mIJRLVUIWLOrikcfteytyWW
* Expire in 0 ms for 6 (transfer 0x55c1ed9dfa90)
* Trying 192.168.1.17...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55c1ed9dfa90)
* Connected to 192.168.1.17 (192.168.1.17) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: self signed certificate in certificate chain
* Closing connection 0
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
user1@server1:~$ curl -k -Lv https://192.168.1.17:9200 -u elastic:mIJRLVUIWLOrikcfteytyWW
* Expire in 0 ms for 6 (transfer 0x559c0820da90)
* Trying 192.168.1.17...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x559c0820da90)
* Connected to 192.168.1.17 (192.168.1.17) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=localhost
* start date: Aug 27 20:46:25 2020 GMT
* expire date: Aug 27 20:46:25 2025 GMT
* issuer: CN=Elastic Certificate Tool Autogenerated CA
* SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
* Server auth using Basic with user 'elastic'
> GET / HTTP/1.1
> Host: 192.168.1.17:9200
> Authorization: Basic ZWxhc3RpYzprMGJtVTc5encxOEdScFBUMTMybw==
> User-Agent: curl/7.64.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 532
<
{
"name" : "server1",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "LFm5WSOZSAuncOVpo-jPLA",
"version" : {
"number" : "7.9.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "a479a2a7fce0389512d6a9361301708b92dff667",
"build_date" : "2020-08-11T21:36:48.204330Z",
"build_snapshot" : false,
"lucene_version" : "8.6.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
* Connection #0 to host 192.168.1.17 left intact
user1@server1:~$
I have to go now, but tomorrow morning I will be available again. Thank you for your help and dedication
This line here is being truncated:
Aug 31 10:07:00 server1 evebox[6500]: 2020-08-31 10:07:00 ERROR evebox::server::main: Failed to get Elasticsearch version, things may not work right: error=request: error sending requ
Can you try running from the command line to get the complete output?
evebox server -c /path/to/your/config
user1@server1:~$ sudo evebox server -c /etc/evebox/evebox.yaml
2020-09-01 08:47:45 INFO evebox::version: This is EveBox version 0.12.0-dev (rev: 3e56627); x86_64-unknown-linux-musl
2020-09-01 08:47:45 INFO evebox::server::main: Using temporary in-memory configuration database
2020-09-01 08:47:45 INFO evebox::sqlite::configrepo: Initializing SQLite database
2020-09-01 08:47:45 INFO evebox::sqlite::configrepo: Updating SQLite database to schema version 1
2020-09-01 08:47:45 ERROR evebox::server::main: Failed to get Elasticsearch version, things may not work right: error=request: error sending request for url (https://127.0.0.1:9200/): error trying to connect: invalid dnsname
2020-09-01 08:47:45 INFO evebox::server::main: Starting server on 127.0.0.1:5636, tls=false
Ok, in your evebox configuration, try specifying "https://localhost:9200" instead of "127.0.0.1".
user1@server1:~$ sudo evebox server -c /etc/evebox/evebox.yaml
[sudo] password for user1:
2020-09-01 11:17:15 INFO evebox::version: This is EveBox version 0.12.0-dev (rev: 3e56627); x86_64-unknown-linux-musl
2020-09-01 11:17:15 INFO evebox::server::main: Using temporary in-memory configuration database
2020-09-01 11:17:15 INFO evebox::sqlite::configrepo: Initializing SQLite database
2020-09-01 11:17:15 INFO evebox::sqlite::configrepo: Updating SQLite database to schema version 1
2020-09-01 11:17:15 INFO evebox::server::main: Found Elasticsearch version 7.9.0 at https://localhost:9200
2020-09-01 11:17:15 INFO evebox::server::main: Starting server on 127.0.0.1:5636, tls=false
Looks like its working. I don't like that the hostnames need to match when using disable certificate checks, but that seems to be the case with the TLS library I use. I will look into that.
Excellent! I could already verify that the Evebox page works and shows recent data, from today. I understand about the TLS verification, but I think it is already a secondary issue, and I trust you will be able to resolve it shortly.
I think I can move on with what I have so far, but if you need help testing and feedback, count on me.
As always, I greatly appreciate your help, interest and above all the promptness of your response.
Closing this issue for now, as during this I did fix an issue connecting to self-signed certificates which was completely broken before. Now it works, but with caveats. Thanks!
I chased down why this requires a hostname instead of an IP address... I use a pure Rust TLS library for this project instead of OpenSSL so I can easily build static binaries, as well as cross compile for ARM - for the best "just works" scenario without running into library issues. This library (rustls) in turn depends on some PKI library that doesn't support connecting to TLS by hostname, while OpenSSL does.
I probably won't switch to OpenSSL for the builds I provide so I can keep the the "just works" approach. But may provide a simple build flag to use it for those who want to build their own EveBox.
Good to know and thank you for sharing! š
Hi,
I am in the process of enabling security options on my ELK nodes. Started enabling X-Pack in ELK and had to reconfigure Evebox. Resolved in https://github.com/jasonish/evebox/issues/143.
Now I am enabling TLS and HTTPS in ELK. Once enabled, Elasticsearch won't accept http connections.
I tried editing evebox.yml as follows, but it did not work.
Apparently it's failing the SSL handshake due to a wrong or missing certificate.
How can I have Evebox to use a certificate generated by Elasticsearch? Or if there is another way, please share the right config to use.
Thank you
ELK log output