search

junegunn / hbase-jruby

A JRuby binding for HBase
MIT License
38 stars 14 forks source link

ssh connection to zookeeper in AWS #27

Closed rockrep closed 11 years ago

rockrep commented 11 years ago

Hi, I'm trying to connect an HBase client instance to an HBase master running in a cluster on AWS EMR. I created the cluster with the command

./elastic-mapreduce --create --hbase --name "My HBase Cluster" --bootstrap-action s3://elasticmapreduce/bootstrap-actions/configure-hbase-daemons --args "--hbase-zookeeper-opts=-Xmx1024m -XX:GCTimeRatio=19,--hbase-master-opts=-Xmx2048m,--hbase-regionserver-opts=-Xmx4096m" --instance-type m1.large --key-pair myawskeypair

I can successfully manually ssh to the instance using AWS CLI tools, where I created the hbase table using hbase shell, and I've opened an SSH tunnel following the guidelines in http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-ssh-tunnel.html. The tunnel has allowed me to access the HBase web services such as the master-status running on port 60010.

Unfortunately, I'm unable to successfully connect with your gem client 

HBase.resolve_dependency! '0.92.0'
@hbase = HBase.new 'hbase.zookeeper.quorum' => '<master public DNS name of cluster>'
@hbase[:mytable]

13/09/27 16:02:46 INFO zookeeper.ClientCnxn: Opening socket connection to server <cluster DNS location>:2181
13/09/27 16:03:07 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Operation timed out
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1041)

Any suggestions appreciated. FWIW, connecting is fine to a localhost running HBase instance.

junegunn commented 11 years ago

Hi, thanks for the report. Unfortunately, I don't have experience with AWS EMR, so I'm not sure if I can help you right now, but I'll look into it.

Can you connect to it with Java? hbase-jruby is just a wrapper for HBase Java API. So if you can do it in Java, I could find a way to make hbase-jruby work as well.

rockrep commented 11 years ago

Looks like the resolution here is to add to associate the public ip address with the internal ec2 domain name in the /etc/hosts file for the hbase-jruby client.