search

dask / hdfs3

A wrapper for libhdfs3 to interact with HDFS from Python
http://hdfs3.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
136 stars 40 forks source link

Is Hdfs3 support hadoop 1.0? #155

Closed kurossssd closed 6 years ago

kurossssd commented 6 years ago

Hello,

I'm getting the following error with hdfs3 0.3.0 , Hadoop 1.x :

from hdfs3 import HDFileSystem hdfs = HDFileSystem ( host = "Hadoop 1.x host', port = 8020 )

Error:

ConnectionError : Connection Failed: HdfsRpcExection: Faild to invoke RPC call "getFsStats" on server
Caused by: HdfsEndOfStream: Read 4 bytes failed from host : End of stream

core-site.xml:

< fs.default.name ><Hadoop 1.x host>

but I can connect with hadoop client and Hadoop 2.x it works well. e.g ) hadoop fs -ls hdfs://Hadoop 1.x host:8020/

Is Hdfs3(libhdfs3) support hadoop 1.0? if it support well, how can I connect my hadoop 1.0 server?

martindurant commented 6 years ago

Honestly, this has not come up, so I don't know. It sounds indeed like it may not be supported. Hadoop 2.x has been around for four years...

kurossssd commented 6 years ago

Okay. thank you for the quick reply