hdfswriter 不支持HDFS的HA

[biansutao]

           "writer": {
                "name": "hdfswriter",
                "parameter": {
                     **"defaultFS": "hdfs://hacluster",**
                    "fileType": "text",
                    "path": "/tmp/datax",
                    "fileName": "sg_ssd_rtu_p",
                    "column": [
                        {
                            "name": "col1",
                            "type": "STRING"
                        },
                        {
                            "name": "col2",
                            "type": "STRING"
                        },
                        {
                            "name": "col3",
                            "type": "STRING"
                        }
                    ],
                    "writeMode": "append",
                    "fieldDelimiter": ","
                }
            }

        }
    ]
}

"defaultFS": "hdfs://hacluster",

这样写法的FS，DataX无法识别和使用。

[biansutao]

现在生产系统已经都使用HA了，DataX是不是考虑支持一下。 谢谢！

[strive-kun]

DataX是支持HA的， "defaultFS": "hdfs://hacluster", "hadoopConfig":{ "dfs.nameservices": "hacluster", "dfs.ha.namenodes.hacluster": "nn1,nn2", "dfs.namenode.rpc-address.hacluster.nn1": "...:8020", "dfs.namenode.rpc-address.hacluster.nn2": "...:8020", "dfs.client.failover.proxy.provider.hacluster": "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider" },

[jiaomb]

参照reader那边改一下重新编译