kzk / webhdfs

Ruby client for Hadoop WebHDFS
Other
81 stars 46 forks source link

Not enough information in request exception #22

Closed tungel closed 8 years ago

tungel commented 8 years ago

My hadoop setup consists of a few datanodes. And the hosts file /etc/hosts supposed to be something like:

IP_ADDRESS_1 hdslave01 name1
IP_ADDRESS_2 hdslave01 name2
IP_ADDRESS_3 hdslave01 name3

Let's say for some reason, some entries in the host file don't exist.

When I try to read a file from HDFS with the below code snippet:

client = WebHDFS::Client.new(@web_host, @web_port, 'user')
file = client.read(@path);

I understand that Hadoop will perform a redirection to the actual datanode upon received the above request. As a result I receive a not so useful error message like:

#<SocketError: getaddrinfo: Name or service not known>

where the actual problem maybe that the name3 host is not defined in my hosts file. Is there a way to construct a more meaningful error message when this exception is raised?

I see that there is no exception handling in the call conn.request and conn.send_request in request method in client_v1.rb. I can submit a pull request later if needed.

Thanks.

tagomoris commented 8 years ago

IMO, Name or service not known is enough meaningful, ... but pull-requests are welcome to fix it, like Failed to connect host ($hostname): Name or service not known or something.

tungel commented 8 years ago

Thanks for the response. In my case, since we have quite a number of datanodes, having the error message pointing out directly which datanode is the problem is very useful.