Append api could not throw WebHDFS::FileNotFoundError?

[zhangzhonglai]

Hi,

It will throw a exception named Errno::EPIPE: Broken pipe if I append the data which is bigger than 3M and the path is not exist.

2.2.1 :058 > @client.append("/tmp/webhdfs_append_test2", "ab" * 512 * 1024 * 3)
Errno::EPIPE: Broken pipe
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/protocol.rb:211:in `write'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/protocol.rb:211:in `write0'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/protocol.rb:185:in `block in write'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/protocol.rb:202:in `writing'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/protocol.rb:184:in `write'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http/generic_request.rb:187:in `send_request_with_body'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http/generic_request.rb:120:in `exec'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:1412:in `block in transport_request'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:1411:in `catch'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:1411:in `transport_request'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:1384:in `request'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:1377:in `block in request'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:853:in `start'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:1375:in `request'
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:1355:in `send_request'
    from /Users/zhangzhonglai/.rvm/gems/ruby-2.2.1@zzl/gems/webhdfs-0.7.0/lib/webhdfs/client_v1.rb:329:in `request'
    from /Users/zhangzhonglai/.rvm/gems/ruby-2.2.1@zzl/gems/webhdfs-0.7.0/lib/webhdfs/client_v1.rb:261:in `operate_requests'
    from /Users/zhangzhonglai/.rvm/gems/ruby-2.2.1@zzl/gems/webhdfs-0.7.0/lib/webhdfs/client_v1.rb:75:in `append'
    from (irb):58
    from /Users/zhangzhonglai/.rvm/rubies/ruby-2.2.1/bin/irb:11:in `<main>'2.2.1 :059 >

What's wrong about that?

[tagomoris]

Sorry for late reply. Are you using http reverse proxies? Software such like that sometimes limits HTTP content body size.

[zhangzhonglai]

Thank you for reply. I didn't configured the http reverse proxies. I'm using the CDH5 and I'm not sure what internal mechanism it use. But you supported me a direction. I will check the configure of that aspect.