search

tagomoris / fluent-plugin-secure-forward

Other
140 stars 30 forks source link

Public CA Cert appears as invalid #27

Closed clu-0x80 closed 8 years ago

clu-0x80 commented 9 years ago

I have attempted to setup a receiver with a public CA cert and it is being deemed as untrusted by both the sender and external utilities such as openssl.

The reciever has a public CA issued cert from godaddy which has a 3 part bundle of intermediate CA certs that is appended into a pem file (key, cert, bundle). The cert is a wildard cert that is in production elsewhere so it is trusted elsewhere. So I feel the problem is that part of the certificate chain is not being sent.

Can an option be made to specify intermediate CAs?

Recv Source Config:

    type secure_forward
    shared_key      TjKjGwwNyLD3McwD4
    self_hostname   logging.company.co
    secure  true
    cert_path       /etc/td-agent/ssl/wildcard_company_co-godaddy-2015.pem
    private_key_path /etc/td-agent/ssl/wildcard_company_co-godaddy-2015.key
    private_key_passphrase 
    authentication  yes
    <user>
            username        compuser
            password        compass
    </user>

Sender Source Config: <match **> type secure_forward shared_key TjKjGwwNyLD3McwD4 self_hostname appserver.company.co secure true enable_strict_verification yes

host logging.company.co # or IP # port 24284 username compuser password compass

Sender td-agent logs: 015-07-31 10:57:30 +0100 [warn]: failed to establish SSL connection error_class=OpenSSL::SSL::SSLError error=# host="logging.company.co" address="123.123.123.123" port=24284

OpenSSL output from domain using this cert and considered trusted: Certificate chain 0 s:/OU=Domain Control Validated/CN=*.company.co i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2 1 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2 i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2 2 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2 i:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority 3 s:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority i:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority

TD-Agent Receiver connection output: openssl sclient -connect logging.company.co:24284 CONNECTED(00000003) depth=0 OU = Domain Control Validated, CN = .company.co verify error:num=20:unable to get local issuer certificate verify return:1 depth=0 OU = Domain Control Validated, CN = .company.co verify error:num=27:certificate not trusted verify return:1 depth=0 OU = Domain Control Validated, CN = .company.co verify error:num=21:unable to verify the first certificate verify return:1 Certificate chain 0 s:/OU=Domain Control Validated/CN=_.company.co i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2

tagomoris commented 9 years ago

@clu-0x80 Intermediate certs should be concatenated into one .pem file and specified from configuration file of secure forward server (input plugin).

clu-0x80 commented 9 years ago

This is exactly how I presented the certs at one point and this still did not work. I tried various combinations.

clu-0x80 commented 9 years ago

OS: Ubuntu 12.04 Plugin: fluent-plugin-secure-forward (0.3.2) TD-Agent Version: 2.2.1-0 TD-Agent installed from PPA

jacknagz commented 8 years ago

@tagomoris I'm also experiencing this same exact problem. My server certificate is concatenated and it still does not work.

tagomoris commented 8 years ago

Can someone paste commands to generate concatenated server cert to reproduce this problem?

jacknagz commented 8 years ago

https://www.digicert.com/ssl-support/pem-ssl-creation.htm

tagomoris commented 8 years ago

Thanks! I'll check it later...

tagomoris commented 8 years ago

I confirmed that concatenated pem files doesn't work well on this plugin. Now i'm fixing it.

tagomoris commented 8 years ago

I've just fixed this problem, fixed and released v0.3.3. This version will handle chained/concatenated PEM certificates from public CAs correctly. @jacknagz @clu-0x80 Could you check this release? Please reopen this issue if you still have problems.

jacknagz commented 8 years ago

Hey @tagomoris , with the new 0.3.3 release, my td-agent fails to start when loading the secure-forward gem. I'm using this configuration:

<source>

  type secure_forward
  bind 0.0.0.0
  port 4443
  secure yes
  self_hostname <omitted_cert_common_name>
  shared_key
  cert_path /etc/td-agent/certs/cert.pem
  private_key_path /etc/td-agent/certs/key.pem
  private_key_passphrase

</source>
repeatedly commented 8 years ago

@jacknagz Please paste your error log. With only configuration, hard to investigate why td-agent fails to start.

jacknagz commented 8 years ago

@repeatedly there's no error log, it just hangs when running # service td-agent start. td-agent.log tells me it hangs after adding source type="secure_forward"

tagomoris commented 8 years ago

@jacknagz Are there any information in debug/trace level logs? https://docs.treasuredata.com/articles/td-agent#debugging

Please tell me about your platform (Linux?) and the version of td-agent.

jacknagz commented 8 years ago

Platform: RHEL 6.4 TD-Agent: 0.12.12

2015-12-01 17:11:41 +0000 [info]: fluent/supervisor.rb:403:read_config: reading config file path="/etc/td-agent/td-agent.conf"
2015-12-01 17:11:42 +0000 [info]: fluent/supervisor.rb:279:supervise: starting fluentd-0.12.12
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered buffer plugin 'file'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered buffer plugin 'memory'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered filter plugin 'grep'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered filter plugin 'record_transformer'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered filter plugin 'stdout'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'debug_agent'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'dummy'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'exec'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'forward'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'gc_stat'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'http'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'monitor_agent'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'object_space'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'status'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'unix'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'syslog'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'tail'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'tcp'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'udp'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'copy'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'exec'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'exec_filter'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'file'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'forward'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'null'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'relabel'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'roundrobin'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'stdout'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'tcp'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'unix'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluent-mixin-config-placeholders' version '0.3.0'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluent-plugin-mongo' version '0.7.10'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluent-plugin-rewrite-tag-filter' version '1.4.1'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluent-plugin-s3' version '0.5.9'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluent-plugin-scribe' version '0.10.14'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluent-plugin-secure-forward' version '0.3.3'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluent-plugin-splunk-ex' version '1.0.2'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluent-plugin-td' version '0.10.27'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluent-plugin-td-monitoring' version '0.2.1'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluent-plugin-webhdfs' version '0.4.1'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluentd' version '0.12.12'
2015-12-01 17:11:42 +0000 [info]: fluent/engine.rb:90:block in configure: gem 'fluentd' version '0.10.61'
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered output plugin 'splunk_ex'
2015-12-01 17:11:42 +0000 [info]: fluent/root_agent.rb:142:add_source: adding source type="secure_forward"
2015-12-01 17:11:42 +0000 [trace]: fluent/plugin.rb:98:register_impl: registered input plugin 'secure_forward'
tagomoris commented 8 years ago

Hmm, weird. I tried this plugin v 0.3.3 with RHEL 6.7 (can't get 6.4 OS image) and td-agent 2.2.1. It works fine.

[tagomoris@td-agent-test ~]$ sudo td-agent-gem install fluent-plugin-secure-forward
WARN: Unresolved specs during Gem::Specification.reset:
      json (>= 1.4.3)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
Fetching: resolve-hostname-0.0.4.gem (100%)
Successfully installed resolve-hostname-0.0.4
Fetching: proxifier-1.0.3.gem (100%)
Successfully installed proxifier-1.0.3
Fetching: fluent-plugin-secure-forward-0.3.3.gem (100%)
Successfully installed fluent-plugin-secure-forward-0.3.3
Parsing documentation for fluent-plugin-secure-forward-0.3.3
Installing ri documentation for fluent-plugin-secure-forward-0.3.3
Parsing documentation for proxifier-1.0.3
Installing ri documentation for proxifier-1.0.3
Parsing documentation for resolve-hostname-0.0.4
Installing ri documentation for resolve-hostname-0.0.4
Done installing documentation for fluent-plugin-secure-forward, proxifier, resolve-hostname after 0 seconds
3 gems installed
[tagomoris@td-agent-test ~]$ sudo vi /etc/td-agent/td-agent.conf 
[tagomoris@td-agent-test ~]$ sudo service td-agent restart
Shutting down td-agent:                                    [  OK  ]
Starting td-agent:                                         [  OK  ]

Logs:

2015-12-02 05:45:19 +0000 [info]: reading config file path="/etc/td-agent/td-agent.conf"
2015-12-02 05:45:19 +0000 [info]: starting fluentd-0.12.12
2015-12-02 05:45:19 +0000 [info]: gem 'fluent-mixin-config-placeholders' version '0.3.0'
2015-12-02 05:45:19 +0000 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
2015-12-02 05:45:19 +0000 [info]: gem 'fluent-plugin-mongo' version '0.7.10'
2015-12-02 05:45:19 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.4.1'
2015-12-02 05:45:19 +0000 [info]: gem 'fluent-plugin-s3' version '0.5.9'
2015-12-02 05:45:19 +0000 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
2015-12-02 05:45:19 +0000 [info]: gem 'fluent-plugin-secure-forward' version '0.3.3'
2015-12-02 05:45:19 +0000 [info]: gem 'fluent-plugin-td' version '0.10.27'
2015-12-02 05:45:19 +0000 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.1'
2015-12-02 05:45:19 +0000 [info]: gem 'fluent-plugin-webhdfs' version '0.4.1'
2015-12-02 05:45:19 +0000 [info]: gem 'fluentd' version '0.12.12'
2015-12-02 05:45:19 +0000 [info]: adding match pattern="td.*.*" type="tdlog"
2015-12-02 05:45:19 +0000 [info]: adding match pattern="debug.**" type="stdout"
2015-12-02 05:45:19 +0000 [info]: adding source type="forward"
2015-12-02 05:45:19 +0000 [info]: adding source type="http"
2015-12-02 05:45:19 +0000 [info]: adding source type="debug_agent"
2015-12-02 05:45:19 +0000 [info]: adding source type="secure_forward"
2015-12-02 05:45:19 +0000 [warn]: 'insecure' mode has vulnerability for man-in-the-middle attacks for clients (output plugins).
2015-12-02 05:45:19 +0000 [info]: using configuration file: <ROOT>
  <match td.*.*>
    type tdlog
    apikey xxxxxx
    auto_create_table 
    buffer_type file
    buffer_path /var/log/td-agent/buffer/td
    <secondary>
      type file
      path /var/log/td-agent/failed_records
      buffer_path /var/log/td-agent/failed_records.*
    </secondary>
  </match>
  <match debug.**>
    type stdout
  </match>
  <source>
    type forward
  </source>
  <source>
    type http
    port 8888
  </source>
  <source>
    type debug_agent
    bind 127.0.0.1
    port 24230
  </source>
  <source>
    @type secure_forward
    port 24224
    self_hostname hoge.pos.com
    secure no
    shared_key xxxxxx
  </source>
</ROOT>
2015-12-02 05:45:19 +0000 [info]: listening fluent socket on 0.0.0.0:24224
2015-12-02 05:45:19 +0000 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"
tagomoris commented 8 years ago

@jacknagz Could you try to re-install td-agent and fluent-plugin-secure-forward once more? Please open the other issue, not add comment on here if you still have problem about it.

jacknagz commented 8 years ago

@tagomoris can you try using a public CA setup?

tagomoris commented 8 years ago

@jacknagz I have no time to do that right now... But I think that your problem isn't related with public CA configurations. If so, Fluentd logs must show setup for ssl sessions and other debug/trace logs before handling certificates. Did you check your configuration without public CA certificates?