Agent auth fails with OCSP

quulah commented 2 years ago

New root CA, intermediate, certificates with OCSP information generated. Agents fail to connect.

Expected Behavior

Agents should authenticate successfully.

Current Behavior

Backend logs say:

Nov 01 19:59:24 <backend> sensu-backend[4027774]: 2021/11/01 19:59:24 [WARNING] error checking revocation via OCSP

I find this interesting, because it is differently formatted than the other logs. And I couldn't find this line in Sensu code. Must come from a library further down the pipes.

On the agents:

Nov 01 22:01:27 <agent> sensu-agent[20014]: {"component":"agent","level":"info","msg":"connecting to backend URL \"wss://<backend>:8081\"","time":"2021-11-01T22:01:27+02:00"}
Nov 01 22:01:27 <agent> sensu-agent[20014]: {"component":"agent","error":"handshake failed with status 400: ","level":"error","msg":"reconnection attempt failed","time":"2021-11-01T22:01:27+02:00"}

From the OCSP error, and the actual message missing after status 400: I've begin thinking that something horrible happens and the backend fails to respond correctly.

I have set up an OCSP responder, tested it with openssl ocsp and some online checkers and it should be fine.

No requests appear in its logs when Sensu is running and states the above error.

Also, openssl s_client seems to be happy with the certs:

openssl s_client -connect <backend>:8081 -key agent.key -cert agent.crt 
...
Verification: OK

Possible Solution

It would probably be good to have an option to control OCSP checks.

The error messages could be more descriptive too.

Steps to Reproduce (for bugs)

Create and configure certificates with AIA information for OCSP (and possibly CA issuers).

Context

We've been running Sensu for a while successfully with an internal root CA signed certificates.

I have now recreated our CA, added an intermediate in between and also added OCSP.

I have followed the instructions, so the chain is configured and the leaf certificate is the first one in the bundle.

Your Environment

Sensu version used (sensuctl, sensu-backend, and/or sensu-agent): 6.5.1 & 6.5.4
Installation method (packages, binaries, docker etc.): .deb packages from packagecloud.io/sensu/stable with sensu-go-ansible
Operating System and version (e.g. Ubuntu 14.04): Backend 20.04, agents 18.04

bitnik commented 1 month ago

@quulah could you solve your issue?

quulah commented 1 month ago

Nope. :(

We've also transitioned away from Sensu for the monitoring since perhaps two years ago, so I am unable to say whether newer versions change the behavior.

On Thu 23. May 2024 at 18.47, Kenan Erdogan @.***> wrote:

@quulah https://github.com/quulah could you solve your issue?

— Reply to this email directly, view it on GitHub https://github.com/sensu/sensu-go/issues/4472#issuecomment-2127465836, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKWJCHHFBZWFUBXEU2XE2DZDYFRHAVCNFSM5HE2HVPKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJSG42DMNJYGM3A . You are receiving this because you were mentioned.Message ID: @.***>

pkerspe commented 1 month ago

We are experiencing the same issue, also could not find a solution after hours of debugging and investigation. We figured though, that the error message "[WARNING] error checking revocation via OCSP" is generated in an external library that is used for the revocation check of the certificate: cfssl package https://github.com/cloudflare/cfssl/

We verified our certificates successfully using openssl against the ocsp URI provided in the issued agent certificates and all is fine. Also the CRL URI from the certificate can be accesse wirhout a problem from the sesnu backend host.

The error logging is unfortunately very poor in this regard in sensu.

sensu / sensu-go