Connection hanging

[Rockyyost]

When trying to create a table, I open the connection and pass the table, like below:

RBHive.connect('hive.server.address', 10_000) do |connection| connection.create_table(table) end

But with Hiveserver, I never get a response, the table doesn't get created, and the connection stays, never disconnecting. It does say that it's Executing Hive Query, and spells out the table creation statement, but that's it.

It's similar with Hiveserver2, only I do get a message about the connection happening. After that, it behaves like Hiveserver.

Thanks!

[andytinycat]

Are you running a Hiveserver or Hiveserver2 endpoint on port 10000? If you try to talk Hiveserver to a Hiveserver2 endpoint, you'll get a hang because the Thrift protocols are not compatible at all. You can only run one kind of Thrift server on a given port (obviously).

If you're using a recent Hive version, you're probably running Hiveserver2. Out of the box (IIRC), Hiveserver2 has SASL enabled - so I bet what's happening here is:

• Hiveserver Thrift connection won't work because you're running Hiveserver2
• Hiveserver2 Thrift connection won't work because out-of-the-box Hiveserver2 uses SASL, and trying to talk to it without SASL will cause a hang.

You can turn SASL off by putting this in your Hive configuration and restarting the Thrift service:

<property>
  <name>hive.server2.authentication</name>
  <value>NOSASL</value>
</property>

[Rockyyost]

Thanks for your help! I added it to the hive-site.xml, but than Thirft didn't want to restart. After removing the line, Thirft still didn't want to restart. I'm using CDH4 with Cloudera Manager and the damn thing is brittle! I was able to get Thrift back up on another server and I'm at the same place as before.

Assuming I'm stuck with SASL, how would I use it with rbhive?

Thanks again for your help!

[andytinycat]

I can't comment on why it wouldn't restart - you need to consult your log files. I don't use Cloudera Manager, so I don't know where to look - I presume the interface has some facility for viewing why the startup failed? If you want to paste the log files here, I could look.

You don't have to be "stuck with SASL" - that's a valid config item (I copy/pasted it from our production cluster); there must be a typo or something odd at your end that's stopping it from starting.

Unfortunately I don't know how to use Hive with SASL as it's not something we've done. The committer who contributed the SASL code may know. But I would strongly advise you not to go down this route without first examining why the Thrift service would not start - check your logs.

[Rockyyost]

No worries on the restart issue, I'm working through that and, like I said, I was able to get HiveServer2 working on another server.

Also, I figure out how to use rbhive with SASL, I just include :transport => :sasl, :sasl_params => {} in the connections options hash parameter, and it works like it should. I was able to create my tables and such.

Thanks for your help Andy!