Open dengshaochun opened 6 years ago
when i use webhdfs with kerberos hadoop cluster
[hdfs@nfjd-hadoop02-node27 ~]$ irb irb(main):001:0> require 'webhdfs' => true irb(main):002:0> require 'gssapi' => true irb(main):003:0> client = WebHDFS::Client.new('nfjd-hadoop02-node169', 50070) => #<WebHDFS::Client:0xfd0e5b6 @ssl_ca_file=nil, @ssl_version=nil, @kerberos_keytab=nil, @doas=nil, @retry_times=1, @httpfs_mode=false, @proxy_port=nil, @ssl_verify_mode=nil, @kerberos=false, @ssl=false, @username=nil, @ssl_cert=nil, @http_headers={}, @proxy_address=nil, @retry_known_errors=false, @port=50070, @host="nfjd-hadoop02-node169", @retry_interval=1, @ssl_key=nil> irb(main):004:0> client.kerberos = true => true irb(main):005:0> client.kerberos_keytab = "/home/hdfs/keytab/hdfs.keytab" => "/home/hdfs/keytab/hdfs.keytab" irb(main):006:0> irb(main):007:0* irb(main):008:0* client.list('/user/dengsc') WebHDFS::IOError: <html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/><title>Error 403 GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails)</title></head><body><h2>HTTP ERROR 403</h2><p>Problem accessing /webhdfs/v1/user/dengsc. Reason:<pre> GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails)</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> </body></html> from /home/hdfs/logstash-5.6.3/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:401:in `request' from /home/hdfs/logstash-5.6.3/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:275:in `operate_requests' from /home/hdfs/logstash-5.6.3/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:138:in `list' from (irb):8:in `evaluate' from org/jruby/RubyKernel.java:1079:in `eval' from org/jruby/RubyKernel.java:1479:in `loop' from org/jruby/RubyKernel.java:1242:in `catch' from org/jruby/RubyKernel.java:1242:in `catch' from /home/hdfs/logstash/vendor/jruby/bin/irb:13:in `(root)'
I am facing the same issue. Providing a keytab file shows the same error.
However, if I do a kinit -kt <MYKEYTAB> before I use this gem, everything works fine (even If don't provide the keytab file later)
kinit -kt <MYKEYTAB>
when i use webhdfs with kerberos hadoop cluster