search

kzk / webhdfs

Ruby client for Hadoop WebHDFS
Other
81 stars 46 forks source link

Adds Knox Support #12

Open att-bdpaas opened 9 years ago

att-bdpaas commented 9 years ago

We are using WebHDFS through a Knox gateway.

This requires:

  1. Specifying the gateway and the cluster to the path
  2. Support for basic auth

Let us know if you have any questions/ feedback about this PR.

Thanks!

tagomoris commented 9 years ago

I'm sorry for late reply. Are there any other Knox gateway users? I cannot find anyone around me. I'll re-consider to merge if any others adds comments for merge.

About patch, I think that we should change API of initialize like below, and change username/doas/proxy_address/proxy_port as deprecated:

def initialize(host='localhost', port=50070, username=nil, doas=nil, proxy_address=nil, proxy_port=nil, options={})
   # check options[:username], options[:doas], ... , options[:knox_path], ...
   # ...
end
tlabeeuw commented 9 years ago

Knox is obviously relatively new, but is definitely shaping up to be the standard way to connect to a secure Kerberized cluster. Both Hortonworks and Cloudera now have it available.

Regarding the initialize method this seems like a decent approach. Though it does seem arbitrary what goes in options and what goes as positioned arguments. Ruby's 2.0 named arguments would solve this but guessing you want your library to support as many Ruby versions as possible.