Mask credentials when logging Kafka producer and consumer configuration

[dkantor]

When OMRS logs the Kafka producer and consumer configuration info, the sasl.jaas.config and ssl.truststore.password properties are in clear text. These properties need to be hidden/masked.

The messages are defined in org.odpi.openmetadata.adapters.eventbus.topic.kafka.KafkaOpenMetadataTopicConnectorAuditCode.SERVICE_PRODUCER_PROPERTIES and SERVICE_CONSUMER_PROPERTIES, and are being logged by org.odpi.openmetadata.adapters.eventbus.topic.kafka.KafkaOpenMetadataEventConsumer and KafkaOpenMetadataEventProducer.

Here is an example of how the information is currently logged. Note that the sasl.jaas.config and ssl.truststore.password properties are in clear text. :

[2/12/19 16:44:26:144 EST] 0000003f SystemOut O {"appname":"wdp-catalog","timestamp":"2019-02-12 16:44:26.144","transaction_ID":"","loglevel":"INFO ","thread":"lt Executor-thread-9","class":"OMRSAuditLog","method":"logRecord","line":"98","message":"1dd78413-6cea-444d-960b-c13e1c79204b_omag OCF-KAFKA-TOPIC-CONNECTOR-0003 The properties passed to the Apache Kafka Consumer for topic open-metadata.repository-services.cohort.dkantor_cohort.OMRSTopic are: {key.deserializer=org.apache.kafka.common.serialization.StringDeserializer, sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username = "Rv0UMcFFz3B4impy" password = "iW8sQV2F9YRzSqdc9dEJKjkDw6uKE54y";, value.deserializer=org.apache.kafka.common.serialization.StringDeserializer, zookeeper.session.timeout.ms=300000, ssl.enabled.protocols=TLSv1.2, ssl.truststore.location=C:\software\ibm-java-sdk-8.0.5.6\jre\lib\security\cacerts, ssl.truststore.password=changeit, enable.auto.commit=true, security.protocol=SASL_SSL, sasl.mechanism=PLAIN, fetch.message.max.bytes=10485760, ssl.protocol=TLSv1.2, group.id=c64c5fb2-3854-4da2-85ac-b2e867cdf4d9, ssl.truststore.type=JKS, zookeeper.sync.time.ms=2000, bootstrap.servers=kafka04-prod02.messagehub.services.us-south.bluemix.net:9093,kafka05-prod02.messagehub.services.us-south.bluemix.net:9093,kafka03-prod02.messagehub.services.us-south.bluemix.net:9093,kafka02-prod02.messagehub.services.us-south.bluemix.net:9093,kafka01-prod02.messagehub.services.us-south.bluemix.net:9093, session.timeout.ms=30000, max.partition.fetch.bytes=10485760, auto.commit.interval.ms=1000}"}

[mandy-chessell]

Fix added to release 1.0 - equivalent update to master is in progress

[hjv333]

@mandy-chessell

Using egeria release-1.0 branch build (from March 04, 2019) with WKC, there are four egeria log entries that are still showing the kafka sasl.jaas.config and ssl.truststore.password info during start of omrs instance.

The attachment "logged_kafka_credentials.txt" has the log entries during omrs instance start that have the kafka credentials.

logged_kafka_credentials.txt

During stop of omrs instance, there are two log entries that have the kafka credentials.

logged_kafka_credentials_during_stop.txt

[planetf1]

I am also seeing this in 1.1 master as of 12 Mar 19.

Here's my start. I have log level set to INFO

https://gist.github.com/a0ffb08e01edf8094a5da948ffb3887b

kafka itself tends to set the value in logs to be [hidden]

[planetf1]

Additionally a REST API call to retrieve configuration also exposes the above secrets ie GET {{baseURL}}/open-metadata/admin-services/users/{{user}}/servers/{{server}}/configuration/

[mandy-chessell]

This fix has only be added to 1.0 at this time - the fix for master is still in my workspace

[mandy-chessell]

It is reasonable that the credentials are return on the GET request because it is the administrator that sets it up