search

acryldata / datahub-actions

DataHub Actions is a framework for responding to changes to your DataHub Metadata Graph in real time.
42 stars 47 forks source link

feat(schema-registry): respect schema_registry_config #39

Closed jsotelo closed 2 years ago

jsotelo commented 2 years ago

We use our own ca certs for our kafka schema registry so we need to configure our "ingestion_executor" with the cert path. For example:

name: "ingestion_executor" 
source:
  type: "kafka"
  config:
    connection:
      bootstrap: ${KAFKA_BOOTSTRAP_SERVER:-localhost:9092}
      schema_registry_url: ${SCHEMA_REGISTRY_URL:-http://localhost:8081}
      schema_registry_config:
        ssl.ca.location: "/usr/local/share/ca-certificates/mycompany/my-certs.crt"

This PR passes these setting to the SchemaRegistryClient