Multi-connector connector

[tmnd1991]

Multi-Connector

Multi-connector can connect to multiple connectors in a hierarchical order. It is useful when schemas are registered on different datastore (i.e. confluent + hbase).

You configure it in the following way:

darwin {
  type = "lazy"
  connector = "multi"
  registrator = "hbase"
  confluent-single-object-encoding: ["confluent"]
  standard-single-object-encoding: ["hbase", "mongo"]
  confluent {
    endpoints: ["http://schema-registry-00:7777", "http://schema-registry-01:7777"]
    max-cached-schemas: 1000
  }
  hbase {
    isSecure: false
    namespace: "DARWIN"
    table: "REPOSITORY"
    coreSite: "/etc/hadoop/conf/core-site.xml"
    hbaseSite: "/etc/hadoop/conf/hbase-site.xml"
  }
  mongo {
    username = "mongo"
    password = "mongo"
    host = ["localhost:12345"]
    database = "test"
    collection = "collection_test"
    timeout = 5000
  }
}

When extracting the schemaId, it will check if the single object encoding is "confluent" or "standard" way. Given that, it will go down the chain of confluent-single-object-encoding or standard-single-object-encoding in order. The first that matches, is the one that will be used.

Register of schema, will work with the connector set as registrator.