christiangalsterer / httpbeat

Elastic Beat to call HTTP endpoints
Apache License 2.0
107 stars 40 forks source link

Panic when using TLS configuration (Go 1.7.4) #15

Closed MarkSonghurst closed 7 years ago

MarkSonghurst commented 7 years ago

I would get a panic if I attempted to use any TLS related configuration when building httpbeats on Go 1.7.4 I'm slightly unsure how the function poller.go convertTLSConfig() could have ever worked, maybe something got tightened up in Go 1.7.4

christiangalsterer commented 7 years ago

Thanks for the contribution. Will look into it the next days/weekend.

MarkSonghurst commented 7 years ago

Can we get a new release/build please? Having the prebuilt RPM is great - thanks.

christiangalsterer commented 7 years ago

I will look into another item #14 on the weekend and depending on the outcome I will cut a release on the weekend.

Mrc0113 commented 7 years ago

@MarkSonghurst Can you share the ssl config that worked for you with your updates?

MarkSonghurst commented 7 years ago
httpbeat:
  # Httpbeat configuration for API hosts.
  urls:
    -
      cron: "@every 10s"
      url: https://localhost:8080/metrics
      method: get
      document_type: httpbeat_api_metrics
      output_format: json
      headers:
        Accept: application/json
      ssl:
        certificate_authorities: ["/etc/pki/tls/certs/logstash-ca.crt"]
        verification_mode: none
    -
      cron: "@every 5s"
      url: https://localhost:8080/alerts
      method: get
      document_type: httpbeat_api_alerts
      output_format: json
      headers:
        Accept: application/json
      ssl:
        certificate_authorities: ["/etc/pki/tls/certs/logstash-ca.crt"]
        verification_mode: none
MarkSonghurst commented 7 years ago

For logstash

#----------------------------- Logstash output --------------------------------
output.logstash:
  # Boolean flag to enable or disable the output module.
  enabled: true  

  # Number of workers per Logstash host.
  worker: 1  

  # Enable SSL support. SSL is automatically enabled, if any SSL setting is set.
  ssl.enabled: true

  # Configure SSL verification mode. If `none` is configured, all server hosts
  # and certificates will be accepted. In this mode, SSL based connections are
  # susceptible to man-in-the-middle attacks. Use only for testing. Default is
  # `full`.
  ssl.verification_mode: none

  # The Logstash hosts
  hosts: ["monitoring.equineregister.co.uk:5044"]

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  certificate_authorities: ["/etc/pki/tls/certs/logstash-ca.crt"]

  # Certificate for TLS client authentication
  certificate: "/etc/pki/tls/certs/logstash-forwarder.crt"

  # Client Certificate Key
  certificate_key: "/etc/pki/tls/private/logstash-forwarder.key"
MarkSonghurst commented 7 years ago

Without my PR I would get an immediate Go panic when the SSL config was loaded, and httpbeat would terminate. I'm really surprised the guys in #14 are not seeing that also.

Anyway, once I fixed the panic, I had some error messages on STDOUT about an invalid cert which led me to use both the verification_mode: none and certificate_authorities settings. I suspect that will solve #14 - however both might not be needed. YMMV.

Incidently, the comment in the logstash section: # Enable SSL support. SSL is automatically enabled, if any SSL setting is set. I found to be untrue. I had to specifically set ssl.enabled: true to make SSL work against logstash.

Mrc0113 commented 7 years ago

Thanks - I can set those properties yet they don't work for me. My suspicion is that the "verification_mode" property doesn't work properly as I can curl the endpoint with the --insecure option which is supposed to be the equivalent of "verification_mode: none" but httpbeat seems to still try to verify the hostname against the common name in the certificate.

Unfortunately the system I am trying to call is creating a self signed certificate that does not have the proper host name...

christiangalsterer commented 7 years ago

Release 3.2.0 now available