search

cube-js / cube

šŸ“Š Cube ā€” Universal semantic layer platform for AI, BI, spreadsheets, and embedded analytics
https://cube.dev
Other
17.97k stars 1.78k forks source link

SparkSQL Connection error: v2.3.2 #297

Open binodsarkar opened 4 years ago

binodsarkar commented 4 years ago

SparkSQL Connection error Spark sql connection through thrift s not working. Please find the error

Error while loading DB schema
Error: TApplicationException: Required field 'client_protocol' is unset!
Struct:TOpenSessionReq(client_protocol:null, username:anonymous, password:)
at Client.OpenSession (.../node_modules/jshs2/lib/HiveConnection.js:57:18)
at client._reqs.(anonymous function) (.../node_modules/thrift/lib/nodejs/lib/thrift/connection.js:151:13)
at exports.Client.TCLIServiceClient.recv_OpenSession (.../node_modules/jshs2/idl/Thrift_0.9.3_Hive_2.1.1/TCLIService.js:2327:12)
at .../node_modules/thrift/lib/nodejs/lib/thrift/connection.js:157:41
at .../node_modules/thrift/lib/nodejs/lib/thrift/buffered_transport.js:48:5
at frames.filter.map.f (.../node_modules/@cubejs-backend/hive-driver/driver/TSaslTransport.js:105:52)
at Array.map (<anonymous>)
at Socket.<anonymous> (.../node_modules/@cubejs-backend/hive-driver/driver/TSaslTransport.js:105:43)
at Socket.emit (events.js:198:13)
at addChunk (_stream_readable.js:287:12)
tromika commented 4 years ago

Thrift_0.9.3_Hive_2.1.1

Seems there is a version mismatch between the client and the thrift server.

binodsarkar commented 4 years ago

Thrift_0.9.3_Hive_2.1.1

Seems there is a version mismatch between the client and the thrift server.

In that case how to upgrade cubejs hive version? Because my warehouse version in 2.3.2 or any idea how to use sparksql jdbc driver?

And I have tested after downgrading hive spark etc on my local m/c but still not working

tromika commented 4 years ago

by default 2.3.4 bundled see https://github.com/cube-js/cube.js/tree/master/packages/cubejs-hive-driver/idl/Hive_2.3.4. So you have to replace the files with the 2.3.2 version inside your cubejs installation.

tromika commented 4 years ago

I guess it doesn't solve your problem but might solve someone's in the future, you can find older hive and thrift idls in https://github.com/imjuni/jshs2/tree/master/idl

binodsarkar commented 4 years ago

by default 2.3.4 bundled see https://github.com/cube-js/cube.js/tree/master/packages/cubejs-hive-driver/idl/Hive_2.3.4. So you have to replace the files with the 2.3.2 version inside your cubejs installation.

Connection error for bug fixing version change?

binodsarkar commented 4 years ago

I guess it doesn't solve your problem but might solve someone's in the future, you can find older hive and thrift idls in https://github.com/imjuni/jshs2/tree/master/idl

Sorry, the exact version 2.3.4 also not resolved my issue. I think I am missing any simple thing? And Any IDEA how to use spark sql jdbc because in my another node application jdbc works fine and I have also seen that there is a driver for sparksql in cubejs cubejs jdbc https://github.com/cube-js/cube.js/blob/fd39c95136b00668026999faad5d19712c7df65d/packages/cubejs-jdbc-driver/driver/JDBCDriver.js#L60

paveltiunov commented 4 years ago

@binodsarkar Hey Binod! Hive is very sensitive with these version stuff. You should have correct IDL in place: https://github.com/cube-js/cube.js/tree/master/packages/cubejs-hive-driver#contributing-missing-hive-protocol.

On JDBC side: unfortunately node-maven isn't able to install it due to it doesn't support variables AFAIK.

CBox commented 4 years ago

Hey, just to make sure what version of SparkSQL is working with CubeJS?

tromika commented 4 years ago

Hi @CBox, we are on 2.3.x and it's working properly. I can't say anything about 2.4.x or 3.x unfortunately but keep in mind for the CubeJs and Spark integration the Hive versions matters.

paveltiunov commented 4 years ago

This issue is currently outside of our roadmap and we're looking for contributions here!