search

k8ssandra / management-api-for-apache-cassandra

RESTful / Secure Management Sidecar for Apache Cassandra
Apache License 2.0
72 stars 51 forks source link

Some useful log messages are missing when MTLS is enabled #42

Open respringer opened 4 years ago

respringer commented 4 years ago

I have noticed in my testing that when MTLS certificates are enabled, there is less logging by the mgmt api. Specifically the "address= url= status=" messages are not logged when MTLS is enabled, and this has made automated test validations more difficult to perform.

The following logs were taken from a pod that was bootstrapping. Notice the missing readiness and liveness probe calls. This is a small example, but there are no "address=foo" messages in log for the entire session for the MTLS example.

Example when MTLS is disabled:

cassandra INFO [epollEventLoopGroup-36-2 2020-10-02 15:47:52,834 Uuids.java:194 - PID obtained through native call to getpid(): 19 cassandra WARN [epollEventLoopGroup-36-2 2020-10-02 15:47:53,400 AbstractBootstrap.java:452 - Unknown channel option 'TCP_NODELAY' for channel '[id: 0x72990ea9' server-system-logger tail: can't open '/var/log/cassandra/system.log': No such file or directory server-system-logger tail: /var/log/cassandra/system.log has appeared; following end of new file

Same setup but with MTLS disabled:

cassandra INFO [nioEventLoopGroup-2-1 2020-10-02 15:58:04,580 Cli.java:617 - address=/10.244.4.5:55580 url=/api/v0/probes/cluster status=200 OK cassandra INFO [nioEventLoopGroup-2-2 2020-10-02 15:58:06,574 Cli.java:617 - address=/10.244.5.1:48544 url=/api/v0/probes/readiness status=200 OK cassandra INFO [nioEventLoopGroup-2-1 2020-10-02 15:58:06,596 Cli.java:617 - address=/10.244.4.5:55606 url=/api/v0/metadata/endpoints status=200 OK cassandra INFO [nioEventLoopGroup-2-2 2020-10-02 15:58:06,622 Cli.java:617 - address=/10.244.4.5:55608 url=/api/v0/ops/seeds/reload status=200 OK cassandra INFO [nioEventLoopGroup-2-1 2020-10-02 15:58:06,649 Cli.java:617 - address=/10.244.4.5:55612 url=/api/v0/probes/cluster status=200 OK server-system-logger tail: can't open '/var/log/cassandra/system.log': No such file or directory server-system-logger tail: /var/log/cassandra/system.log has appeared; following end of new file

Please enable the "address=foo" log messages, and any additional useful messages that are being omitted, when MTLS is enabled.

┆Issue is synchronized with this Jira Story by Unito ┆Issue Number: MAPI-60

burmanm commented 2 years ago

@emerkle826 This is indeed true still. I hit this bug today when trying to improve MTLS testing.. there's no drain lines for example, so I can't verify if the drain is called (the MTLS has different preStop process than non-MTLS, which is why testing feels sort of important)