DTStack / chunjun

A data integration framework
https://dtstack.github.io/chunjun/
Apache License 2.0
4.01k stars 1.69k forks source link

Kafka 组件 SSL 认证时,认证文件如何加载? #1075

Closed Aiurvie closed 2 years ago

Aiurvie commented 2 years ago

Search before asking

Description

请问一下,现在纯钧 kafka 模块支持 ssl 认证吗?我看源码中好像没有做支持,如果要做这样的支持,有没有可以参考的东西?

我疑惑的是 SSL 认证时 .jks 文件如何加载。 平时写普通 Flink 程序的时候,我是改了 Kafka Client 读取文件的方式,把 .jks 文件给打到程序包里了,但是 Flink SQL Connector 不能这么做。

期待解惑。

Code of Conduct

kyo-tom commented 2 years ago

You can add ssl config into producerSettings. For example:

{
  "job": {
    "content": [
      {
        "reader": {
          "name": "mysqlreader",
          "parameter": {
            "column": [
              {
                "name": "id",
                "type": "int"
              },
              {
                "name": "raw_date",
                "type": "string",
                "value": "2014-12-12 14:24:16"
              }
            ],
            "username": "root",
            "password": "root",
            "connection": [
              {
                "jdbcUrl": [
                  "jdbc:mysql://localhost:3306/test?useSSL=false"
                ],
                "table": [
                  "sync_ods"
                ]
              }
            ]
          }
        },
        "writer": {
          "name": "kafkasink",
          "parameter": {
            "tableFields": [
              "id",
              "raw_date"
            ],
            "topic": "cx",
            "producerSettings": {
              "auto.commit.enable": "false",
              "bootstrap.servers": "localhost:9092",
              "ssl.keystore.location": "keystore_path",
              "ssl.keystore.password": "keystore_pass",
              "sssl.keystore.type": "jks",
              "ssl.truststore.location": "truststore_path",
              "ssl.truststore.password": "truststore_pass",
              "sssl.truststore.type": "jks",
              "ssl.key.password": "key_pass",
              "ssl.endpoint.identification.algorithm": "https"
            }
          }
        }
      }
    ],
    "setting": {
      "restore": {
        "isRestore": true,
        "isStream": true
      },
      "speed": {
        "readerChannel": 1,
        "writerChannel": 1
      }
    }
  }
}
yanghuaiGit commented 2 years ago

good

yanghuaiGit commented 2 years ago

This issue has been around for too long, so close it and reopen it if necessary.