hdfs-site.xml with multiple clusters

colinmarc / hdfs

A native go client for HDFS

MIT License

1.37k stars 341 forks source link

hdfs-site.xml with multiple clusters #225

Open eldadts opened 4 years ago

eldadts commented 4 years ago

When using HADOOP_CONF_DIR or HADOOP_HOME, how to specify to which cluster you want to connect when the hdfs-site.xml file contain multiple clusters, and it's notfs.defaultFS?

eldadts commented 4 years ago

after checking the source code there is a bug in hadoopConf.go -> when returning name nodes it actually drops the defaultFS (dfs.ha.namenodes.) and just keeps all the namenodes of ALL the clusters (dfs.namenode.rpc-address) ...

example files are attached hdfs-files.zip

j4ns8i commented 4 years ago

@xtrimf I've submitted a PR that should fix your issues FYI ^

eldadts commented 4 years ago

@j4ns8i tnx! when will this be merged so I can test it?

j4ns8i commented 4 years ago

@xtrimf not sure unfortunately. i'm unaffiliated with this project so it'll depend on collaboration with the maintainer(s) to get it merged