Kafka to BigQuery with SSL

[matheusjerico]

How I can use kafka to bigquery template (https://github.com/GoogleCloudPlatform/DataflowTemplates/tree/master/v2/kafka-to-bigquery or https://github.com/GoogleCloudPlatform/java-docs-samples/tree/master/dataflow/flex-templates/kafka_to_bigquery) with SSL Kafka?

I didn't find any configuration to pass SSL files (truststore/keystore) to template.

How I can make this configuration?

[sabhyankar]

We are working on releasing a Kafka template that supports additional security capabilities including SSL and Kerberos. This should be available in this repo later in this quarter.

[matheusjerico]

@sabhyankar Thanks!

[orensham]

@matheusjerico https://github.com/doitintl/dataflow-kafka-to-bq

[Ozzyz]

@sabhyankar any updates on this? Has it been implemented? Found this, but not sure if it covers OPs needs

[dodywicahyo]

Any update? Just wonder to change default security.protocol=PLAINTEXT to SSL

[nosovk]

@dodywicahyo do you managed to solve an issue somehow?

[jasmas]

https://cloud.google.com/dataflow/docs/guides/templates/ssl-certificates

[nosovk]

https://cloud.google.com/dataflow/docs/guides/templates/ssl-certificates

as I see - no kafka support

[jasmas]

https://cloud.google.com/dataflow/docs/guides/templates/ssl-certificates

as I see - no kafka support

This tells you how to store and use the cert chain with dataflow. Everything else about how to use it is documented in org.apache.kafka.