forward3d / rbhive

Ruby gem for querying Apache Hive
http://www.forward3d.com
MIT License
98 stars 74 forks source link

Empty resultset using :sasl on HIve 0.13 and 0.14 #28

Closed m1racoli closed 9 years ago

m1racoli commented 9 years ago

Using SASL the query doesn't return rows:

RBHive.tcli_connect(host,10_000,{ logger: Logger.new(STDOUT), hive_version: 13, transport: :sasl, sasl_params: {username: 'hive'} }) do |connection|
  res = connection.fetch(query)
  puts res.column_names.inspect
  puts res.first.inspect
end

This is the output:

I, [2015-02-12T07:06:43.543538 #1333]  INFO -- : Executing Hive Query: SHOW TABLES
[:tab_name]
nil
=> nil

Also async execution doesn't yield anything:

RBHive.tcli_connect(host,10_000,{ logger: Logger.new(STDOUT), hive_version: 13, transport: :sasl, sasl_params: {username: 'hive'} }) do |connection|
  hash = connection.async_execute(query2)
  state = connection.async_state(hash)
  while state == :running do
      puts state
      sleep(2)
    state = connection.async_state(hash)
  end  
  puts state  
  puts connection.async_fetch(hash)
  connection.async_close_session(hash)
end
I, [2015-02-12T07:09:54.103844 #1333]  INFO -- : Executing query asynchronously: SHOW TABLES
2
finished
2
=> <Hive2::Thrift::TCloseSessionResp status:<Hive2::Thrift::TStatus statusCode:SUCCESS_STATUS (0)>>
andytinycat commented 9 years ago

I'm not sure if this is relevant, but the hive_version: 13 doesn't work properly at the moment. The protocol changed so that the data is return in a columnar-based rather than a row-based orientation. My memory is fuzzy, but I'm pretty sure this manifests as getting nil for rows.

You can still speak the 0.12 version of the protocol to 0.13 - try again with hive_version: 12.

To fix it, the gem would need some fairly significant reworking. At the moment that's something we aren't likely to do for the foreseeable future, simply because we've decided to stay on Hive 0.12 due to show-stopper bugs in 0.13 and above, and moved our querying platform to Facebook's Presto.

m1racoli commented 9 years ago

Thank you, using hive_version: 12 did resolve the problem.