search

cloudera / hue

Open source SQL Query Assistant service for Databases/Warehouses
https://cloudera.com
Apache License 2.0
1.17k stars 366 forks source link

The Hive query editor fails to translate a variable when using a path as a parameter #395

Closed ArcTheMaster closed 8 years ago

ArcTheMaster commented 8 years ago

Hi, I'm using Hue 3.9.0 and Hive 2.0.1. I run into a strange behavior since the last week. In the query editor, I create the following query: USE ${DBNAME}; ADD JAR ${LOCATION1}; ADD JAR ${LOCATION2};

Then I click on "execute". Hue asks me to specify my parameters for the query. After setting my parameters I execute the query. Unfortunately the query fails.

On the UI I have: Your query has the following error(s): Error while processing statement: java.net.URISyntaxException: Illegal character in path at index 1: ${LOCATION1}

On the error.log I have: [28/Jun/2016 11:39:07 +0200] dbms ERROR Bad status for request TExecuteStatementReq(confOverlay={}, sessionHandle=TSessionHandle(sessionId=THandleIdentifier(secret='\x03\x86\xc8STLHd\xa9\xfdn\x04\xa6\xf7o\xbf', guid='\xfb|\x95o\xe3xJ\n\x8e\xe8mGt\xc0\xd2\x87')), runAsync=True, statement='ADD JAR ${LOCATION1}'): TExecuteStatementResp(status=TStatus(errorCode=1, errorMessage='Error while processing statement: java.net.URISyntaxException: Illegal character in path at index 1: ${LOCATION1}', sqlState=None, infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Error while processing statement: java.net.URISyntaxException: Illegal character in path at index 1: ${LOCATION1}:27:26', 'org.apache.hive.service.cli.operation.Operation:toSQLException:Operation.java:367', 'org.apache.hive.service.cli.operation.HiveCommandOperation:runInternal:HiveCommandOperation.java:118', 'org.apache.hive.service.cli.operation.Operation:run:Operation.java:309', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementInternal:HiveSessionImpl.java:450', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementAsync:HiveSessionImpl.java:427', 'sun.reflect.GeneratedMethodAccessor16:invoke::-1', 'sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43', 'java.lang.reflect.Method:invoke:Method.java:606', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78', 'org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36', 'org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63', 'java.security.AccessController:doPrivileged:AccessController.java:-2', 'javax.security.auth.Subject:doAs:Subject.java:415', 'org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1657', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59', 'com.sun.proxy.$Proxy33:executeStatementAsync::-1', 'org.apache.hive.service.cli.CLIService:executeStatementAsync:CLIService.java:259', 'org.apache.hive.service.cli.thrift.ThriftCLIService:ExecuteStatement:ThriftCLIService.java:504', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1317', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1302', 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', 'org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor:process:HadoopThriftAuthBridge.java:714', 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286', 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', 'java.lang.Thread:run:Thread.java:745'], statusCode=3), operationHandle=None) Traceback (most recent call last): File "/opt/application/Hue/hue-3.9.0/hue/apps/beeswax/src/beeswax/server/dbms.py", line 559, in execute_and_watch handle = self.client.query(query, query_history.statement_number) File "/opt/application/Hue/hue-3.9.0/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 875, in query return self._client.execute_async_query(query, statement) File "/opt/application/Hue/hue-3.9.0/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 655, in execute_async_query return self.execute_async_statement(statement=query_statement, confOverlay=configuration) File "/opt/application/Hue/hue-3.9.0/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 673, in execute_async_statement res = self.call(self._client.ExecuteStatement, req) File "/opt/application/Hue/hue-3.9.0/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 574, in call raise QueryServerException(Exception('Bad status for request %s:\n%s' % (req, res)), message=message) QueryServerException: Bad status for request TExecuteStatementReq(confOverlay={}, sessionHandle=TSessionHandle(sessionId=THandleIdentifier(secret='\x03\x86\xc8STLHd\xa9\xfdn\x04\xa6\xf7o\xbf', guid='\xfb|\x95o\xe3xJ\n\x8e\xe8mGt\xc0\xd2\x87')), runAsync=True, statement='ADD JAR ${LOCATION1}'): TExecuteStatementResp(status=TStatus(errorCode=1, errorMessage='Error while processing statement: java.net.URISyntaxException: Illegal character in path at index 1: ${LOCATION1}', sqlState=None, infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Error while processing statement: java.net.URISyntaxException: Illegal character in path at index 1: ${LOCATION1}:27:26', 'org.apache.hive.service.cli.operation.Operation:toSQLException:Operation.java:367', 'org.apache.hive.service.cli.operation.HiveCommandOperation:runInternal:HiveCommandOperation.java:118', 'org.apache.hive.service.cli.operation.Operation:run:Operation.java:309', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementInternal:HiveSessionImpl.java:450', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementAsync:HiveSessionImpl.java:427', 'sun.reflect.GeneratedMethodAccessor16:invoke::-1', 'sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43', 'java.lang.reflect.Method:invoke:Method.java:606', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78', 'org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36', 'org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63', 'java.security.AccessController:doPrivileged:AccessController.java:-2', 'javax.security.auth.Subject:doAs:Subject.java:415', 'org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1657', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59', 'com.sun.proxy.$Proxy33:executeStatementAsync::-1', 'org.apache.hive.service.cli.CLIService:executeStatementAsync:CLIService.java:259', 'org.apache.hive.service.cli.thrift.ThriftCLIService:ExecuteStatement:ThriftCLIService.java:504', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1317', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1302', 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', 'org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor:process:HadoopThriftAuthBridge.java:714', 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286', 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', 'java.lang.Thread:run:Thread.java:745'], statusCode=3), operationHandle=None) [28/Jun/2016 11:39:07 +0200] api ERROR error in <function watch_query_refresh_json at 0x7f5ee545b668> Traceback (most recent call last): File "/opt/application/Hue/hue-3.9.0/hue/apps/beeswax/src/beeswax/api.py", line 52, in decorator return view_fn(request, *args, **kwargs) File "/opt/application/Hue/hue-3.9.0/hue/apps/beeswax/src/beeswax/api.py", line 187, in watch_query_refresh_json raise ex QueryServerException: Bad status for request TExecuteStatementReq(confOverlay={}, sessionHandle=TSessionHandle(sessionId=THandleIdentifier(secret='\x03\x86\xc8STLHd\xa9\xfdn\x04\xa6\xf7o\xbf', guid='\xfb|\x95o\xe3xJ\n\x8e\xe8mGt\xc0\xd2\x87')), runAsync=True, statement='ADD JAR ${LOCATION1}'): TExecuteStatementResp(status=TStatus(errorCode=1, errorMessage='Error while processing statement: java.net.URISyntaxException: Illegal character in path at index 1: ${LOCATION1}', sqlState=None, infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Error while processing statement: java.net.URISyntaxException: Illegal character in path at index 1: ${LOCATION1}:27:26', 'org.apache.hive.service.cli.operation.Operation:toSQLException:Operation.java:367', 'org.apache.hive.service.cli.operation.HiveCommandOperation:runInternal:HiveCommandOperation.java:118', 'org.apache.hive.service.cli.operation.Operation:run:Operation.java:309', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementInternal:HiveSessionImpl.java:450', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementAsync:HiveSessionImpl.java:427', 'sun.reflect.GeneratedMethodAccessor16:invoke::-1', 'sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43', 'java.lang.reflect.Method:invoke:Method.java:606', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78', 'org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36', 'org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63', 'java.security.AccessController:doPrivileged:AccessController.java:-2', 'javax.security.auth.Subject:doAs:Subject.java:415', 'org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1657', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59', 'com.sun.proxy.$Proxy33:executeStatementAsync::-1', 'org.apache.hive.service.cli.CLIService:executeStatementAsync:CLIService.java:259', 'org.apache.hive.service.cli.thrift.ThriftCLIService:ExecuteStatement:ThriftCLIService.java:504', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1317', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1302', 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', 'org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor:process:HadoopThriftAuthBridge.java:714', 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286', 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', 'java.lang.Thread:run:Thread.java:745'], statusCode=3), operationHandle=None)

This error happens only when the parameter is a path (local or HDFS) and if you have more than one variable declared in your script. In that case, Hue fails to translate the variable to its real value.

romainr commented 8 years ago

This is fixed in Hue 3.10!

ArcTheMaster commented 8 years ago

Cool thanks for your reply!