TCLI Connection Object Question

[gjackson12]

I would like to re-use a connection object.

I have something like this:

connection = RBHive::TCLIConnection.new(server, port, options) connection.open connection.open_session

I use the connection object to execute a query, and then:

connection.close_session connection.close

If I try to re-use the same connection object it just seems to hang when I try to open a new session:

connection.open connection.open_session

Any idea what could be happening and how to get around it? (my knowledge of Thrift is limited so I apologize in advance if this is a dumb question).

[andytinycat]

No such thing as a dumb question!

What you're doing isn't valid (and really the gem should prevent you from doing it). Hangs indicate you've sent a message to the server it doesn't like, or wasn't expecting, so the gem is sitting waiting for a reply. In this case, I don't think it expects you to try to reuse a connection object.

Rather than reusing the object, reuse the session. So create your connection and open the session:

connection = RBHive::TCLIConnection.new(server, port, options)
connection.open
connection.open_session
... store this somewhere, in a class or whatever ...

Then you can just reach in and use this connection again and again as long as you never close it.

This is risky, though - if you exit and never close the session you opened, the Hive Thrift service will keep it open along with any memory allocated to it. If you kept opening connections and not closing them, the Hive service will eventually run out of memory. This is made worse if you use asynchronous querying,

So I guess the question is why you want to do it this way? It's much safer to spin up connections as and when you need them, using the block form:

RBHive.tcli_connect('hive.server.address', 10_000) do |connection|
  connection.fetch('SHOW TABLES')
end

[gjackson12]

Thank you for answering my question.

I was hoping to create a connection object upon initialization of my application and just re-use that same object, seemed less wasteful.

I did not reuse the session because I figured that would expire eventually.

I think I will just go with the block form instead. Thank you again for your help. Love this gem.