elastic appears to be using http even when https is specified

[binkleym]

I'm using Elastic 1.1.0 on Windows using R 3.6.3 to try to connect to an ES server. Here is the script I'm using:

library(elastic)

es <- connect(host = "elk1.vampire",
              port = "9200",
              errors = "complete",
              transport_scheme = "https",
              user = "elastic",
              pwd  = "not_the_real_password",
              cainfo = "ACCRE/ca.crt")

Search(es, "depot")

On the client side, I see the error:

Error in curl::curl_fetch_memory(x$url$url, handle = x$url$handle) : 
  Empty reply from server

On the server side, I see the error:

[2020-08-05T10:23:22,245][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [elk1.vampire] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/10.0.11.151:9200, remoteAddress=/10.0.11.40:32911}

So even though I'm specifiying "https" as the transport_scheme and giving it a valid CA crt, it seems to be trying http to connect to the ES server.

Any suggstions? Thanks.

[sckott]

I don't see any reason why that would occur looking at the elastic package code.

Can you turn on verbose curl output crul::set_verbose() then run it again and see if the curl output gives any useful information

[binkleym]

I got a little closer to figuring it out. I enabled debugging and caught the following message:

• SSL certificate problem: unable to get local issuer certificate
• Closing connection 14 Error in curl::curl_fetch_memory(x$url$url, handle = x$url$handle) : SSL certificate problem: unable to get local issuer certificate

It appears that it's insufficient to specify the ca file using elastic's "connect(cainfo = "this_ca.crt"). It appears I have to add it to the OS's CA list by:

1) cd /usr/local/share/ca-certificates 2) mkdir my_folder && cd my_folder 3) cp /tmp/ca.crt . 4) sudo update-ca-certificates

Now it's going farther before erroring out. I'm still seeing "http" instead of "https" on the server side so I'm curious if it's falling back to http if it can't negotiate a successful https connection.

• Trying 10.0.11.151:9200...

• TCP_NODELAY set

• Connected to elk1.vampire (10.0.11.151) port 9200 (#0)

• ALPN, offering h2

• ALPN, offering http/1.1

• successfully set certificate verify locations:

• CAfile: /etc/ssl/certs/ca-certificates.crt CApath: /etc/ssl/certs

• SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

• ALPN, server did not agree to a protocol

• Server certificate:

• subject: CN=elk1.vampire

• start date: Jan 31 18:18:59 2020 GMT

• expire date: Jan 15 18:18:59 2023 GMT

• common name: elk1.vampire (matched)

• issuer: CN=ACCRE Internal CA

• SSL certificate verify ok.

  HEAD / HTTP/1.1 Host: elk1.vampire:9200 User-Agent: libcurl/7.68.0 r-curl/4.3 crul/1.0.0 Accept: / Accept-Encoding: deflate, gzip, br

• Mark bundle as not supporting multiuse < HTTP/1.1 401 Unauthorized < WWW-Authenticate: Bearer realm="security" < WWW-Authenticate: ApiKey < WWW-Authenticate: Basic realm="security" charset="UTF-8" < content-type: application/json; charset=UTF-8 < content-length: 459 <

• Connection #0 to host elk1.vampire left intact

[sckott]

Yes, makes sense that curl would fall back to http if https does not work.

How does your elasticsearch instance expect the API key? As a header? Maybe the API key isn't being set right? If relevant, this package doesn't support any of the paid Elasticsearch security features as I wasn't able to get access to those to test

[sckott]

closing inactivity