search

eventuate-foundation / eventuate-cdc

Other
72 stars 31 forks source link

KafkaHealthCheck needs to be configurable #60

Open cer opened 4 years ago

cer commented 4 years ago

https://github.com/eventuate-foundation/eventuate-cdc/blob/441f5214f7c7718f9cc3975c2c1d05d5e1e3e150/eventuate-local-java-cdc-connector-unified/src/main/java/io/eventuate/local/unified/cdc/pipeline/common/health/KafkaHealthCheck.java#L22

cer commented 4 years ago

from user - https://eventuate-users.slack.com/archives/C3L9MNMQC/p1592269450123800

It is properties like sasl jaas config, security protocol (to change to SASL_SSL), ssl endpoint identification algorithm (to change to https) and sasl mechanism. The ideal would be to have it configurable the same way through ENV/Spring variable as currently done for producers/consumers
silarr commented 4 years ago

To help and make it clear, here is an example (full names) of parameters for consumer and producer passed through Env variables: EVENTUATE_LOCAL_KAFKA_CONSUMER_PROPERTIES_SASL_MECHANISM: PLAIN EVENTUATE_LOCAL_KAFKA_CONSUMER_PROPERTIES_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: https EVENTUATE_LOCAL_KAFKA_CONSUMER_PROPERTIES_SASL_JAAS_CONFIG: org.apache.kafka.common.security.plain.PlainLoginModule required username="Q4TARSDCREBSDSDGRE" password="Fscjer00sBDLwbrPSOPSmQdKLT0psSDTSDfRI6pgrAxfABD09aQzDdre2SDrfesd+Ug"; EVENTUATE_LOCAL_KAFKA_CONSUMER_PROPERTIES_SECURITY_PROTOCOL: SASL_SSL EVENTUATE_LOCAL_KAFKA_CONSUMER_PROPERTIES_BOOTSTRAP_SERVERS: pkc-6ksd5.us-east1.gcp.confluent.cloud:9092

EVENTUATE_LOCAL_KAFKA_PRODUCER_PROPERTIES_SASL_MECHANISM: PLAIN EVENTUATE_LOCAL_KAFKA_PRODUCER_PROPERTIES_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: https EVENTUATE_LOCAL_KAFKA_PRODUCER_PROPERTIES_SASL_JAAS_CONFIG: org.apache.kafka.common.security.plain.PlainLoginModule required username="Q4TARSDCREBSDSDGRE" password="Fscjer00sBDLwbrPSOPSmQdKLT0psSDTSDfRI6pgrAxfABD09aQzDdre2SDrfesd"; EVENTUATE_LOCAL_KAFKA_PRODUCER_PROPERTIES_SECURITY_PROTOCOL: SASL_SSL EVENTUATE_LOCAL_KAFKA_PRODUCER_PROPERTIES_BOOTSTRAP_SERVERS: pkc-6ksd5.us-east1.gcp.confluent.cloud:9092

cer commented 4 years ago

@dartartem Does the CDC now support the configuration described in https://github.com/eventuate-foundation/eventuate-cdc/issues/60#issuecomment-654468516 ?

dartartem commented 4 years ago

@cer, yes, it supports generic properties, but prefix is different: EVENTUATE_LOCAL_KAFKA_CONSUMER_SASL_MECHANISM: PLAIN

cer commented 4 years ago

What about producer properties?

dartartem commented 4 years ago

@cer, kafka healthcheck does not use producer. But cdc producer already supports generic properties: EVENTUATE_LOCALKAFKAPRODUCER_SASL_MECHANISM: PLAIN