nferrario
commented
1 year ago Same here!
Open VitaliyFedorov opened 1 year ago
nferrario
commented
1 year ago Same here!
linxt20
commented
1 year ago I meet the same problem
giuturtu
commented
1 year ago Same here!
raulbla
commented
1 year ago I can confirm we have the same issue. Instead the query from sql tab is working fine
0xfcryowbk2ca65z
commented
1 year ago Same here!
MarioDurantez
commented
1 year ago I met a similar problem with an Athena datasource. I found that superset 2.1.0 uses SQLAlchemy's get_view_names() here. The pyAthena version suggested in the docs did not implement this call, and the NotImplementedError was being caught and supressed by superset code.
I solved by upgrading pyAthena to >=2.0.0, which already implements get_view_names(). Unfortunately, the most recent Impala driver for sqlAlchemy does not implement this call, only get_table_names().
Currently the only solution seems to be downgrading Superset to 2.0.1.
huw0
commented
1 year ago Thanks @MarioDurantez for this pointer.
I've raised a PR which stubs this method - cloudera/impyla#511. For now, I'm manually patching this file as part of our Docker build pipeline.
chathawee
commented
1 year ago @huw0 Could you please provide the solution. I have tried to add this fuction to superset/db_engine_specs/impala.py but it didn't work. Please suggest.
VitaliyFedorov
commented
1 year ago @huw0 Could you please provide the solution. I have tried to add this fuction to superset/db_engine_specs/impala.py but it didn't work. Please suggest.
I've just build impyla with his PR and install it via pip in superset dockerfile. @huw0 patch impyla code, not superset
rusackas
commented
9 months ago Is this still an issue people are facing in Superset 3.x?
pyroman30
commented
7 months ago Same here!
rusackas
commented
7 months ago @pyroman30 which version of Superset are you using, and which version of the Impala driver? If "same here" means you're using the old versions, as reported in the initial report, then you need to update both pieces of software.
pyroman30
commented
7 months ago @rusackas I'm using helm chart version 2.0.1 and impyla version 0.18.0. I can't figure out how to view the impala driver version?
dkrat7
commented
7 months ago @pyroman30 get_view_name support was added in impyla==0.19.0. You should try with that version.
pyroman30
commented
7 months ago @dkrat7 thanks for the answer, unfortunately it didn't work
pyroman30
commented
7 months ago 2024-04-04 06:57:22,363:DEBUG:impala.hiveserver2:Connecting to HiveServer2 some_host:21050 with GSSAPI authentication mechanism
2024-04-04 06:57:22,363:DEBUG:impala._thrift_api:get_socket: host=some_host port=21050 use_ssl=False ca_cert=None
2024-04-04 06:57:22,363:DEBUG:impala.hiveserver2:sock=<thrift.transport.TSocket.TSocket object at 0x7f46c4e4e790>
2024-04-04 06:57:22,364:DEBUG:impala._thrift_api:get_transport: socket=<thrift.transport.TSocket.TSocket object at 0x7f46c4e4e790> host=some_host kerberos_service_name=impala auth_mechanism=GSSAPI user=None password=fuggetaboutit
2024-04-04 06:57:22,454:DEBUG:impala.hiveserver2:transport=<thrift_sasl.TSaslClientTransport object at 0x7f4615d06280> protocol=<thrift.protocol.TBinaryProtocol.TBinaryProtocolAccelerated object at 0x7f46c4e57190> service=<impala._thrift_gen.ImpalaService.ImpalaHiveServer2Service.Client object at 0x7f46c4e4eca0>
2024-04-04 06:57:22,454:DEBUG:impala.hiveserver2:HiveServer2Connection(service=<impala.hiveserver2.HS2Service object at 0x7f46c4de18e0>, default_db=p_datalake)
2024-04-04 06:57:22,455:DEBUG:impala.hiveserver2:Closing HS2 connection
2024-04-04 06:57:22,455:DEBUG:impala.hiveserver2:close_service: client=<impala._thrift_gen.ImpalaService.ImpalaHiveServer2Service.Client object at 0x7f46c4e4eca0>
2024-04-04 06:57:22,456:DEBUG:impala.hiveserver2:Connecting to HiveServer2 some_host:21050 with GSSAPI authentication mechanism
2024-04-04 06:57:22,456:DEBUG:impala._thrift_api:get_socket: host=some_host port=21050 use_ssl=False ca_cert=None
2024-04-04 06:57:22,456:DEBUG:impala.hiveserver2:sock=<thrift.transport.TSocket.TSocket object at 0x7f46c4e4e760>
2024-04-04 06:57:22,456:DEBUG:impala._thrift_api:get_transport: socket=<thrift.transport.TSocket.TSocket object at 0x7f46c4e4e760> host=some_host kerberos_service_name=impala auth_mechanism=GSSAPI user=None password=fuggetaboutit
2024-04-04 06:57:22,517:DEBUG:impala.hiveserver2:transport=<thrift_sasl.TSaslClientTransport object at 0x7f46c4e4e8b0> protocol=<thrift.protocol.TBinaryProtocol.TBinaryProtocolAccelerated object at 0x7f46c4e4e790> service=<impala._thrift_gen.ImpalaService.ImpalaHiveServer2Service.Client object at 0x7f46c4de1940>
2024-04-04 06:57:22,517:DEBUG:impala.hiveserver2:HiveServer2Connection(service=<impala.hiveserver2.HS2Service object at 0x7f46c4de18e0>, default_db=p_datalake)
2024-04-04 06:57:22,517:DEBUG:impala.hiveserver2:Getting a cursor (Impala session)
2024-04-04 06:57:22,518:DEBUG:impala.hiveserver2:.cursor(): getting new session_handle
2024-04-04 06:57:22,518:DEBUG:impala.hiveserver2:OpenSession: req=TOpenSessionReq(client_protocol=5, username='root', password=None, configuration={})
2024-04-04 06:57:22,518:DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=3)
2024-04-04 06:57:22,519:DEBUG:impala.hiveserver2:Transport opened
2024-04-04 06:57:22,521:DEBUG:impala.hiveserver2:OpenSession: resp=TOpenSessionResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None), serverProtocolVersion=5, sessionHandle=TSessionHandle(sessionId=THandleIdentifier(guid=b'\xa7\xd8\x8f\x00w+B\x0c\x91\xf2\xc9\xa8\xde\xfcS\xd1', secret=b'\x81\xfc\xa8"\xda@I\x8b\xa8\xa2a\xc4b\xcce"')), configuration={'ABORT_ON_DEFAULT_LIMIT_EXCEEDED': '', 'ABORT_ON_ERROR': '0', 'ALLOW_ERASURE_CODED_FILES': '0', 'ALLOW_UNSUPPORTED_FORMATS': '', 'APPX_COUNT_DISTINCT': '0', 'BATCH_SIZE': '0', 'BUFFER_POOL_LIMIT': '', 'CLIENT_IDENTIFIER': '', 'COMPRESSION_CODEC': '', 'COMPUTE_STATS_MIN_SAMPLE_SIZE': '1073741824', 'CPU_LIMIT_S': '0', 'DEBUG_ACTION': '', 'DECIMAL_V2': '1', 'DEFAULT_HINTS_INSERT_STATEMENT': '', 'DEFAULT_JOIN_DISTRIBUTION_MODE': 'BROADCAST', 'DEFAULT_ORDER_BY_LIMIT': '', 'DEFAULT_SPILLABLE_BUFFER_SIZE': '2097152', 'DISABLE_CACHED_READS': '', 'DISABLE_CODEGEN': '0', 'DISABLE_CODEGEN_ROWS_THRESHOLD': '50000', 'DISABLE_OUTERMOST_TOPN': '0', 'DISABLE_ROW_RUNTIME_FILTERING': '0', 'DISABLE_STREAMING_PREAGGREGATIONS': '0', 'DISABLE_UNSAFE_SPILLS': '0', 'ENABLE_EXPR_REWRITES': '1', 'EXEC_SINGLE_NODE_ROWS_THRESHOLD': '100', 'EXEC_TIME_LIMIT_S': '0', 'EXPLAIN_LEVEL': 'STANDARD', 'HBASE_CACHE_BLOCKS': '0', 'HBASE_CACHING': '0', 'IDLE_SESSION_TIMEOUT': '1800', 'KUDU_READ_MODE': 'DEFAULT', 'MAX_ERRORS': '100', 'MAX_IO_BUFFERS': '', 'MAX_MEM_ESTIMATE_FOR_ADMISSION': '0', 'MAX_NUM_RUNTIME_FILTERS': '10', 'MAX_ROW_SIZE': '524288', 'MAX_SCAN_RANGE_LENGTH': '0', 'MEM_LIMIT': '0', 'MIN_SPILLABLE_BUFFER_SIZE': '65536', 'MT_DOP': '', 'NUM_NODES': '0', 'NUM_REMOTE_EXECUTOR_CANDIDATES': '3', 'NUM_ROWS_PRODUCED_LIMIT': '0', 'NUM_SCANNER_THREADS': '0', 'OPTIMIZE_PARTITION_KEY_SCANS': '0', 'PARQUET_ANNOTATE_STRINGS_UTF8': '0', 'PARQUET_ARRAY_RESOLUTION': 'THREE_LEVEL', 'PARQUET_DICTIONARY_FILTERING': '1', 'PARQUET_FALLBACK_SCHEMA_RESOLUTION': 'POSITION', 'PARQUET_FILE_SIZE': '0', 'PARQUET_READ_STATISTICS': '1', 'PLANNER_TESTCASE_MODE': '0', 'PREFETCH_MODE': 'HT_BUCKET', 'QUERY_TIMEOUT_S': '0', 'REPLICA_PREFERENCE': 'CACHE_LOCAL', 'REQUEST_POOL': '', 'RESERVATION_REQUEST_TIMEOUT': '', 'RESOURCE_TRACE_RATIO': '0.000000', 'RM_INITIAL_MEM': '', 'RUNTIME_BLOOM_FILTER_SIZE': '1048576', 'RUNTIME_FILTER_MAX_SIZE': '16777216', 'RUNTIME_FILTER_MIN_SIZE': '1048576', 'RUNTIME_FILTER_MODE': 'GLOBAL', 'RUNTIME_FILTER_WAIT_TIME_MS': '0', 'S3_SKIP_INSERT_STAGING': '1', 'SCAN_BYTES_LIMIT': '0', 'SCAN_NODE_CODEGEN_THRESHOLD': '', 'SCHEDULE_RANDOM_REPLICA': '0', 'SCRATCH_LIMIT': '-1', 'SEQ_COMPRESSION_MODE': '', 'SHUFFLE_DISTINCT_EXPRS': '1', 'STRICT_MODE': '0', 'SYNC_DDL': '0', 'THREAD_RESERVATION_AGGREGATE_LIMIT': '0', 'THREAD_RESERVATION_LIMIT': '3000', 'TIMEZONE': '', 'TOPN_BYTES_LIMIT': '536870912', 'V_CPU_CORES': '', 'http_addr': 'some_host:25000'})
2024-04-04 06:57:22,521:DEBUG:impala.hiveserver2:HiveServer2Cursor(service=<impala.hiveserver2.HS2Service object at 0x7f46c4de18e0>, session_handle=TSessionHandle(sessionId=THandleIdentifier(guid=b'\xa7\xd8\x8f\x00w+B\x0c\x91\xf2\xc9\xa8\xde\xfcS\xd1', secret=b'\x81\xfc\xa8"\xda@I\x8b\xa8\xa2a\xc4b\xcce"')), default_config={'ABORT_ON_DEFAULT_LIMIT_EXCEEDED': '', 'ABORT_ON_ERROR': '0', 'ALLOW_ERASURE_CODED_FILES': '0', 'ALLOW_UNSUPPORTED_FORMATS': '', 'APPX_COUNT_DISTINCT': '0', 'BATCH_SIZE': '0', 'BUFFER_POOL_LIMIT': '', 'CLIENT_IDENTIFIER': '', 'COMPRESSION_CODEC': '', 'COMPUTE_STATS_MIN_SAMPLE_SIZE': '1073741824', 'CPU_LIMIT_S': '0', 'DEBUG_ACTION': '', 'DECIMAL_V2': '1', 'DEFAULT_HINTS_INSERT_STATEMENT': '', 'DEFAULT_JOIN_DISTRIBUTION_MODE': 'BROADCAST', 'DEFAULT_ORDER_BY_LIMIT': '', 'DEFAULT_SPILLABLE_BUFFER_SIZE': '2097152', 'DISABLE_CACHED_READS': '', 'DISABLE_CODEGEN': '0', 'DISABLE_CODEGEN_ROWS_THRESHOLD': '50000', 'DISABLE_OUTERMOST_TOPN': '0', 'DISABLE_ROW_RUNTIME_FILTERING': '0', 'DISABLE_STREAMING_PREAGGREGATIONS': '0', 'DISABLE_UNSAFE_SPILLS': '0', 'ENABLE_EXPR_REWRITES': '1', 'EXEC_SINGLE_NODE_ROWS_THRESHOLD': '100', 'EXEC_TIME_LIMIT_S': '0', 'EXPLAIN_LEVEL': 'STANDARD', 'HBASE_CACHE_BLOCKS': '0', 'HBASE_CACHING': '0', 'IDLE_SESSION_TIMEOUT': '1800', 'KUDU_READ_MODE': 'DEFAULT', 'MAX_ERRORS': '100', 'MAX_IO_BUFFERS': '', 'MAX_MEM_ESTIMATE_FOR_ADMISSION': '0', 'MAX_NUM_RUNTIME_FILTERS': '10', 'MAX_ROW_SIZE': '524288', 'MAX_SCAN_RANGE_LENGTH': '0', 'MEM_LIMIT': '0', 'MIN_SPILLABLE_BUFFER_SIZE': '65536', 'MT_DOP': '', 'NUM_NODES': '0', 'NUM_REMOTE_EXECUTOR_CANDIDATES': '3', 'NUM_ROWS_PRODUCED_LIMIT': '0', 'NUM_SCANNER_THREADS': '0', 'OPTIMIZE_PARTITION_KEY_SCANS': '0', 'PARQUET_ANNOTATE_STRINGS_UTF8': '0', 'PARQUET_ARRAY_RESOLUTION': 'THREE_LEVEL', 'PARQUET_DICTIONARY_FILTERING': '1', 'PARQUET_FALLBACK_SCHEMA_RESOLUTION': 'POSITION', 'PARQUET_FILE_SIZE': '0', 'PARQUET_READ_STATISTICS': '1', 'PLANNER_TESTCASE_MODE': '0', 'PREFETCH_MODE': 'HT_BUCKET', 'QUERY_TIMEOUT_S': '0', 'REPLICA_PREFERENCE': 'CACHE_LOCAL', 'REQUEST_POOL': '', 'RESERVATION_REQUEST_TIMEOUT': '', 'RESOURCE_TRACE_RATIO': '0.000000', 'RM_INITIAL_MEM': '', 'RUNTIME_BLOOM_FILTER_SIZE': '1048576', 'RUNTIME_FILTER_MAX_SIZE': '16777216', 'RUNTIME_FILTER_MIN_SIZE': '1048576', 'RUNTIME_FILTER_MODE': 'GLOBAL', 'RUNTIME_FILTER_WAIT_TIME_MS': '0', 'S3_SKIP_INSERT_STAGING': '1', 'SCAN_BYTES_LIMIT': '0', 'SCAN_NODE_CODEGEN_THRESHOLD': '', 'SCHEDULE_RANDOM_REPLICA': '0', 'SCRATCH_LIMIT': '-1', 'SEQ_COMPRESSION_MODE': '', 'SHUFFLE_DISTINCT_EXPRS': '1', 'STRICT_MODE': '0', 'SYNC_DDL': '0', 'THREAD_RESERVATION_AGGREGATE_LIMIT': '0', 'THREAD_RESERVATION_LIMIT': '3000', 'TIMEZONE': '', 'TOPN_BYTES_LIMIT': '536870912', 'V_CPU_CORES': '', 'http_addr': 'some_host:25000'}, hs2_protocol_version=5)
2024-04-04 06:57:22,522:INFO:impala.hiveserver2:Using database p_datalake as default
2024-04-04 06:57:22,525:DEBUG:impala.hiveserver2:Executing query USE p_datalake
2024-04-04 06:57:22,525:DEBUG:impala.hiveserver2:_execute_async: self._buffer=Batch() self._description=None self._last_operation_active=False self._last_operation=None
2024-04-04 06:57:22,525:DEBUG:impala.hiveserver2:_reset_state: Resetting cursor state
2024-04-04 06:57:22,525:DEBUG:impala.hiveserver2:_execute_async: self._buffer=Batch() self._description=None self._last_operation_active=False self._last_operation=None
2024-04-04 06:57:22,525:DEBUG:impala.hiveserver2:ExecuteStatement: req=TExecuteStatementReq(sessionHandle=TSessionHandle(sessionId=THandleIdentifier(guid=b'\xa7\xd8\x8f\x00w+B\x0c\x91\xf2\xc9\xa8\xde\xfcS\xd1', secret=b'\x81\xfc\xa8"\xda@I\x8b\xa8\xa2a\xc4b\xcce"')), statement='USE p_datalake', confOverlay=None, runAsync=True)
2024-04-04 06:57:22,525:DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=3)
2024-04-04 06:57:22,525:DEBUG:impala.hiveserver2:Transport opened
2024-04-04 06:57:22,561:DEBUG:impala.hiveserver2:ExecuteStatement: resp=TExecuteStatementResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None), operationHandle=TOperationHandle(operationId=THandleIdentifier(guid=b'O\xef\x89Q\x9f\xd8Ct\x00\x00\x00\x00\xd6\x8b\xa1\xb3', secret=b'\x81\xfc\xa8"\xda@I\x8b\xa8\xa2a\xc4b\xcce"'), operationType=0, hasResultSet=False, modifiedRowCount=None))
2024-04-04 06:57:22,561:DEBUG:impala.hiveserver2:_execute_async: self._buffer=Batch() self._description=None self._last_operation_active=True self._last_operation=TOperationHandle(operationId=THandleIdentifier(guid=b'O\xef\x89Q\x9f\xd8Ct\x00\x00\x00\x00\xd6\x8b\xa1\xb3', secret=b'\x81\xfc\xa8"\xda@I\x8b\xa8\xa2a\xc4b\xcce"'), operationType=0, hasResultSet=False, modifiedRowCount=None)
2024-04-04 06:57:22,561:DEBUG:impala.hiveserver2:Waiting for query to finish
2024-04-04 06:57:22,561:DEBUG:impala.hiveserver2:GetOperationStatus: req=TGetOperationStatusReq(operationHandle=TOperationHandle(operationId=THandleIdentifier(guid=b'O\xef\x89Q\x9f\xd8Ct\x00\x00\x00\x00\xd6\x8b\xa1\xb3', secret=b'\x81\xfc\xa8"\xda@I\x8b\xa8\xa2a\xc4b\xcce"'), operationType=0, hasResultSet=False, modifiedRowCount=None))
2024-04-04 06:57:22,562:DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=3)
2024-04-04 06:57:22,562:DEBUG:impala.hiveserver2:Transport opened
2024-04-04 06:57:22,563:DEBUG:impala.hiveserver2:GetOperationStatus: resp=TGetOperationStatusResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None), operationState=2, sqlState=None, errorCode=None, errorMessage=None, hasResultSet=None)
2024-04-04 06:57:22,563:DEBUG:impala.hiveserver2:_wait_to_finish: waited 0.0019729137420654297 seconds so far
2024-04-04 06:57:22,564:DEBUG:impala.hiveserver2:Query finished
2024-04-04 06:57:22,564:DEBUG:impala.hiveserver2:Executing query SHOW SCHEMAS
2024-04-04 06:57:22,564:DEBUG:impala.hiveserver2:_execute_async: self._buffer=Batch() self._description=None self._last_operation_active=True self._last_operation=TOperationHandle(operationId=THandleIdentifier(guid=b'O\xef\x89Q\x9f\xd8Ct\x00\x00\x00\x00\xd6\x8b\xa1\xb3', secret=b'\x81\xfc\xa8"\xda@I\x8b\xa8\xa2a\xc4b\xcce"'), operationType=0, hasResultSet=False, modifiedRowCount=None)
2024-04-04 06:57:22,565:DEBUG:impala.hiveserver2:_reset_state: Resetting cursor state
2024-04-04 06:57:22,565:DEBUG:impala.hiveserver2:CloseOperation: req=TCloseOperationReq(operationHandle=TOperationHandle(operationId=THandleIdentifier(guid=b'O\xef\x89Q\x9f\xd8Ct\x00\x00\x00\x00\xd6\x8b\xa1\xb3', secret=b'\x81\xfc\xa8"\xda@I\x8b\xa8\xa2a\xc4b\xcce"'), operationType=0, hasResultSet=False, modifiedRowCount=None))
2024-04-04 06:57:22,565:DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=3)
2024-04-04 06:57:22,565:DEBUG:impala.hiveserver2:Transport opened
2024-04-04 06:57:22,567:DEBUG:impala.hiveserver2:CloseOperation: resp=TCloseOperationResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None))
2024-04-04 06:57:22,567:DEBUG:impala.hiveserver2:_execute_async: self._buffer=Batch() self._description=None self._last_operation_active=False self._last_operation=None
2024-04-04 06:57:22,567:DEBUG:impala.hiveserver2:ExecuteStatement: req=TExecuteStatementReq(sessionHandle=TSessionHandle(sessionId=THandleIdentifier(guid=b'\xa7\xd8\x8f\x00w+B\x0c\x91\xf2\xc9\xa8\xde\xfcS\xd1', secret=b'\x81\xfc\xa8"\xda@I\x8b\xa8\xa2a\xc4b\xcce"')), statement='SHOW SCHEMAS', confOverlay=None, runAsync=True)
2024-04-04 06:57:22,568:DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=3)
2024-04-04 06:57:22,568:DEBUG:impala.hiveserver2:Transport opened
2024-04-04 06:58:22,306:DEBUG:impala.hiveserver2:Connecting to HiveServer2 some_host:21050 with GSSAPI authentication mechanism
2024-04-04 06:58:22,306:DEBUG:impala._thrift_api:get_socket: host=some_host port=21050 use_ssl=False ca_cert=None
2024-04-04 06:58:22,306:DEBUG:impala.hiveserver2:sock=<thrift.transport.TSocket.TSocket object at 0x7f4615de19a0>
2024-04-04 06:58:22,306:DEBUG:impala._thrift_api:get_transport: socket=<thrift.transport.TSocket.TSocket object at 0x7f4615de19a0> host=some_host kerberos_service_name=impala auth_mechanism=GSSAPI user=None password=fuggetaboutit
2024-04-04 06:58:22,431:DEBUG:impala.hiveserver2:transport=<thrift_sasl.TSaslClientTransport object at 0x7f4615de1b20> protocol=<thrift.protocol.TBinaryProtocol.TBinaryProtocolAccelerated object at 0x7f4615de1430> service=<impala._thrift_gen.ImpalaService.ImpalaHiveServer2Service.Client object at 0x7f4615de1340>
2024-04-04 06:58:22,431:DEBUG:impala.hiveserver2:HiveServer2Connection(service=<impala.hiveserver2.HS2Service object at 0x7f4615de14c0>, default_db=p_datalake)
2024-04-04 06:58:22,432:DEBUG:impala.hiveserver2:Closing HS2 connection
2024-04-04 06:58:22,432:DEBUG:impala.hiveserver2:close_service: client=<impala._thrift_gen.ImpalaService.ImpalaHiveServer2Service.Client object at 0x7f4615de1340>
2024-04-04 06:58:22,432:DEBUG:impala.hiveserver2:Connecting to HiveServer2 some_host:21050 with GSSAPI authentication mechanism
2024-04-04 06:58:22,432:DEBUG:impala._thrift_api:get_socket: host=some_host port=21050 use_ssl=False ca_cert=None
2024-04-04 06:58:22,432:DEBUG:impala.hiveserver2:sock=<thrift.transport.TSocket.TSocket object at 0x7f4615de14c0>
2024-04-04 06:58:22,432:DEBUG:impala._thrift_api:get_transport: socket=<thrift.transport.TSocket.TSocket object at 0x7f4615de14c0> host=some_host kerberos_service_name=impala auth_mechanism=GSSAPI user=None password=fuggetaboutit
2024-04-04 06:58:22,536:DEBUG:impala.hiveserver2:transport=<thrift_sasl.TSaslClientTransport object at 0x7f4615de1970> protocol=<thrift.protocol.TBinaryProtocol.TBinaryProtocolAccelerated object at 0x7f4615de1e20> service=<impala._thrift_gen.ImpalaService.ImpalaHiveServer2Service.Client object at 0x7f4615de1a00>
2024-04-04 06:58:22,537:DEBUG:impala.hiveserver2:HiveServer2Connection(service=<impala.hiveserver2.HS2Service object at 0x7f4615de1880>, default_db=p_datalake)
2024-04-04 06:58:22,537:DEBUG:impala.hiveserver2:Getting a cursor (Impala session)
2024-04-04 06:58:22,537:DEBUG:impala.hiveserver2:.cursor(): getting new session_handle
2024-04-04 06:58:22,537:DEBUG:impala.hiveserver2:OpenSession: req=TOpenSessionReq(client_protocol=5, username='root', password=None, configuration={})
2024-04-04 06:58:22,537:DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=3)
2024-04-04 06:58:22,537:DEBUG:impala.hiveserver2:Transport opened
2024-04-04 06:58:22,539:DEBUG:impala.hiveserver2:OpenSession: resp=TOpenSessionResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None), serverProtocolVersion=5, sessionHandle=TSessionHandle(sessionId=THandleIdentifier(guid=b"\x01c\xbd\x84'\xedG^\xb3\xf1y\x13V\xf1\xa6\x91", secret=b'\x1b\xeb\xec\xab\x15\x85B\xec\xa6\xd1x3\xcf\xb1w/')), configuration={'ABORT_ON_DEFAULT_LIMIT_EXCEEDED': '', 'ABORT_ON_ERROR': '0', 'ALLOW_ERASURE_CODED_FILES': '0', 'ALLOW_UNSUPPORTED_FORMATS': '', 'APPX_COUNT_DISTINCT': '0', 'BATCH_SIZE': '0', 'BUFFER_POOL_LIMIT': '', 'CLIENT_IDENTIFIER': '', 'COMPRESSION_CODEC': '', 'COMPUTE_STATS_MIN_SAMPLE_SIZE': '1073741824', 'CPU_LIMIT_S': '0', 'DEBUG_ACTION': '', 'DECIMAL_V2': '1', 'DEFAULT_HINTS_INSERT_STATEMENT': '', 'DEFAULT_JOIN_DISTRIBUTION_MODE': 'BROADCAST', 'DEFAULT_ORDER_BY_LIMIT': '', 'DEFAULT_SPILLABLE_BUFFER_SIZE': '2097152', 'DISABLE_CACHED_READS': '', 'DISABLE_CODEGEN': '0', 'DISABLE_CODEGEN_ROWS_THRESHOLD': '50000', 'DISABLE_OUTERMOST_TOPN': '0', 'DISABLE_ROW_RUNTIME_FILTERING': '0', 'DISABLE_STREAMING_PREAGGREGATIONS': '0', 'DISABLE_UNSAFE_SPILLS': '0', 'ENABLE_EXPR_REWRITES': '1', 'EXEC_SINGLE_NODE_ROWS_THRESHOLD': '100', 'EXEC_TIME_LIMIT_S': '0', 'EXPLAIN_LEVEL': 'STANDARD', 'HBASE_CACHE_BLOCKS': '0', 'HBASE_CACHING': '0', 'IDLE_SESSION_TIMEOUT': '1800', 'KUDU_READ_MODE': 'DEFAULT', 'MAX_ERRORS': '100', 'MAX_IO_BUFFERS': '', 'MAX_MEM_ESTIMATE_FOR_ADMISSION': '0', 'MAX_NUM_RUNTIME_FILTERS': '10', 'MAX_ROW_SIZE': '524288', 'MAX_SCAN_RANGE_LENGTH': '0', 'MEM_LIMIT': '0', 'MIN_SPILLABLE_BUFFER_SIZE': '65536', 'MT_DOP': '', 'NUM_NODES': '0', 'NUM_REMOTE_EXECUTOR_CANDIDATES': '3', 'NUM_ROWS_PRODUCED_LIMIT': '0', 'NUM_SCANNER_THREADS': '0', 'OPTIMIZE_PARTITION_KEY_SCANS': '0', 'PARQUET_ANNOTATE_STRINGS_UTF8': '0', 'PARQUET_ARRAY_RESOLUTION': 'THREE_LEVEL', 'PARQUET_DICTIONARY_FILTERING': '1', 'PARQUET_FALLBACK_SCHEMA_RESOLUTION': 'POSITION', 'PARQUET_FILE_SIZE': '0', 'PARQUET_READ_STATISTICS': '1', 'PLANNER_TESTCASE_MODE': '0', 'PREFETCH_MODE': 'HT_BUCKET', 'QUERY_TIMEOUT_S': '0', 'REPLICA_PREFERENCE': 'CACHE_LOCAL', 'REQUEST_POOL': '', 'RESERVATION_REQUEST_TIMEOUT': '', 'RESOURCE_TRACE_RATIO': '0.000000', 'RM_INITIAL_MEM': '', 'RUNTIME_BLOOM_FILTER_SIZE': '1048576', 'RUNTIME_FILTER_MAX_SIZE': '16777216', 'RUNTIME_FILTER_MIN_SIZE': '1048576', 'RUNTIME_FILTER_MODE': 'GLOBAL', 'RUNTIME_FILTER_WAIT_TIME_MS': '0', 'S3_SKIP_INSERT_STAGING': '1', 'SCAN_BYTES_LIMIT': '0', 'SCAN_NODE_CODEGEN_THRESHOLD': '', 'SCHEDULE_RANDOM_REPLICA': '0', 'SCRATCH_LIMIT': '-1', 'SEQ_COMPRESSION_MODE': '', 'SHUFFLE_DISTINCT_EXPRS': '1', 'STRICT_MODE': '0', 'SYNC_DDL': '0', 'THREAD_RESERVATION_AGGREGATE_LIMIT': '0', 'THREAD_RESERVATION_LIMIT': '3000', 'TIMEZONE': '', 'TOPN_BYTES_LIMIT': '536870912', 'V_CPU_CORES': '', 'http_addr': 'some_host:25000'})
2024-04-04 06:58:22,539:DEBUG:impala.hiveserver2:HiveServer2Cursor(service=<impala.hiveserver2.HS2Service object at 0x7f4615de1880>, session_handle=TSessionHandle(sessionId=THandleIdentifier(guid=b"\x01c\xbd\x84'\xedG^\xb3\xf1y\x13V\xf1\xa6\x91", secret=b'\x1b\xeb\xec\xab\x15\x85B\xec\xa6\xd1x3\xcf\xb1w/')), default_config={'ABORT_ON_DEFAULT_LIMIT_EXCEEDED': '', 'ABORT_ON_ERROR': '0', 'ALLOW_ERASURE_CODED_FILES': '0', 'ALLOW_UNSUPPORTED_FORMATS': '', 'APPX_COUNT_DISTINCT': '0', 'BATCH_SIZE': '0', 'BUFFER_POOL_LIMIT': '', 'CLIENT_IDENTIFIER': '', 'COMPRESSION_CODEC': '', 'COMPUTE_STATS_MIN_SAMPLE_SIZE': '1073741824', 'CPU_LIMIT_S': '0', 'DEBUG_ACTION': '', 'DECIMAL_V2': '1', 'DEFAULT_HINTS_INSERT_STATEMENT': '', 'DEFAULT_JOIN_DISTRIBUTION_MODE': 'BROADCAST', 'DEFAULT_ORDER_BY_LIMIT': '', 'DEFAULT_SPILLABLE_BUFFER_SIZE': '2097152', 'DISABLE_CACHED_READS': '', 'DISABLE_CODEGEN': '0', 'DISABLE_CODEGEN_ROWS_THRESHOLD': '50000', 'DISABLE_OUTERMOST_TOPN': '0', 'DISABLE_ROW_RUNTIME_FILTERING': '0', 'DISABLE_STREAMING_PREAGGREGATIONS': '0', 'DISABLE_UNSAFE_SPILLS': '0', 'ENABLE_EXPR_REWRITES': '1', 'EXEC_SINGLE_NODE_ROWS_THRESHOLD': '100', 'EXEC_TIME_LIMIT_S': '0', 'EXPLAIN_LEVEL': 'STANDARD', 'HBASE_CACHE_BLOCKS': '0', 'HBASE_CACHING': '0', 'IDLE_SESSION_TIMEOUT': '1800', 'KUDU_READ_MODE': 'DEFAULT', 'MAX_ERRORS': '100', 'MAX_IO_BUFFERS': '', 'MAX_MEM_ESTIMATE_FOR_ADMISSION': '0', 'MAX_NUM_RUNTIME_FILTERS': '10', 'MAX_ROW_SIZE': '524288', 'MAX_SCAN_RANGE_LENGTH': '0', 'MEM_LIMIT': '0', 'MIN_SPILLABLE_BUFFER_SIZE': '65536', 'MT_DOP': '', 'NUM_NODES': '0', 'NUM_REMOTE_EXECUTOR_CANDIDATES': '3', 'NUM_ROWS_PRODUCED_LIMIT': '0', 'NUM_SCANNER_THREADS': '0', 'OPTIMIZE_PARTITION_KEY_SCANS': '0', 'PARQUET_ANNOTATE_STRINGS_UTF8': '0', 'PARQUET_ARRAY_RESOLUTION': 'THREE_LEVEL', 'PARQUET_DICTIONARY_FILTERING': '1', 'PARQUET_FALLBACK_SCHEMA_RESOLUTION': 'POSITION', 'PARQUET_FILE_SIZE': '0', 'PARQUET_READ_STATISTICS': '1', 'PLANNER_TESTCASE_MODE': '0', 'PREFETCH_MODE': 'HT_BUCKET', 'QUERY_TIMEOUT_S': '0', 'REPLICA_PREFERENCE': 'CACHE_LOCAL', 'REQUEST_POOL': '', 'RESERVATION_REQUEST_TIMEOUT': '', 'RESOURCE_TRACE_RATIO': '0.000000', 'RM_INITIAL_MEM': '', 'RUNTIME_BLOOM_FILTER_SIZE': '1048576', 'RUNTIME_FILTER_MAX_SIZE': '16777216', 'RUNTIME_FILTER_MIN_SIZE': '1048576', 'RUNTIME_FILTER_MODE': 'GLOBAL', 'RUNTIME_FILTER_WAIT_TIME_MS': '0', 'S3_SKIP_INSERT_STAGING': '1', 'SCAN_BYTES_LIMIT': '0', 'SCAN_NODE_CODEGEN_THRESHOLD': '', 'SCHEDULE_RANDOM_REPLICA': '0', 'SCRATCH_LIMIT': '-1', 'SEQ_COMPRESSION_MODE': '', 'SHUFFLE_DISTINCT_EXPRS': '1', 'STRICT_MODE': '0', 'SYNC_DDL': '0', 'THREAD_RESERVATION_AGGREGATE_LIMIT': '0', 'THREAD_RESERVATION_LIMIT': '3000', 'TIMEZONE': '', 'TOPN_BYTES_LIMIT': '536870912', 'V_CPU_CORES': '', 'http_addr': 'some_host:25000'}, hs2_protocol_version=5)
2024-04-04 06:58:22,539:INFO:impala.hiveserver2:Using database p_datalake as default
2024-04-04 06:58:22,539:DEBUG:impala.hiveserver2:Executing query USE p_datalake
2024-04-04 06:58:22,540:DEBUG:impala.hiveserver2:_execute_async: self._buffer=Batch() self._description=None self._last_operation_active=False self._last_operation=None
2024-04-04 06:58:22,540:DEBUG:impala.hiveserver2:_reset_state: Resetting cursor state
2024-04-04 06:58:22,540:DEBUG:impala.hiveserver2:_execute_async: self._buffer=Batch() self._description=None self._last_operation_active=False self._last_operation=None
2024-04-04 06:58:22,540:DEBUG:impala.hiveserver2:ExecuteStatement: req=TExecuteStatementReq(sessionHandle=TSessionHandle(sessionId=THandleIdentifier(guid=b"\x01c\xbd\x84'\xedG^\xb3\xf1y\x13V\xf1\xa6\x91", secret=b'\x1b\xeb\xec\xab\x15\x85B\xec\xa6\xd1x3\xcf\xb1w/')), statement='USE p_datalake', confOverlay=None, runAsync=True)
2024-04-04 06:58:22,540:DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=3)
2024-04-04 06:58:22,540:DEBUG:impala.hiveserver2:Transport opened
2024-04-04 06:58:22,575:DEBUG:impala.hiveserver2:ExecuteStatement: resp=TExecuteStatementResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None), operationHandle=TOperationHandle(operationId=THandleIdentifier(guid=b'\x08.\x1c\x0c\xfe\x1cEn\x00\x00\x00\x00\xa1\xbe%', secret=b'\x1b\xeb\xec\xab\x15\x85B\xec\xa6\xd1x3\xcf\xb1w/'), operationType=0, hasResultSet=False, modifiedRowCount=None)) 2024-04-04 06:58:22,576:DEBUG:impala.hiveserver2:_execute_async: self._buffer=Batch() self._description=None self._last_operation_active=True self._last_operation=TOperationHandle(operationId=THandleIdentifier(guid=b'\x08.\x1c\x0c\xfe\x1cEn\x00\x00\x00\x00\xa1\xbe%', secret=b'\x1b\xeb\xec\xab\x15\x85B\xec\xa6\xd1x3\xcf\xb1w/'), operationType=0, hasResultSet=False, modifiedRowCount=None)
2024-04-04 06:58:22,576:DEBUG:impala.hiveserver2:Waiting for query to finish
2024-04-04 06:58:22,576:DEBUG:impala.hiveserver2:GetOperationStatus: req=TGetOperationStatusReq(operationHandle=TOperationHandle(operationId=THandleIdentifier(guid=b'\x08.\x1c\x0c\xfe\x1cEn\x00\x00\x00\x00\xa1\xbe%', secret=b'\x1b\xeb\xec\xab\x15\x85B\xec\xa6\xd1x3\xcf\xb1w/'), operationType=0, hasResultSet=False, modifiedRowCount=None)) 2024-04-04 06:58:22,576:DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=3) 2024-04-04 06:58:22,576:DEBUG:impala.hiveserver2:Transport opened 2024-04-04 06:58:22,578:DEBUG:impala.hiveserver2:GetOperationStatus: resp=TGetOperationStatusResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None), operationState=2, sqlState=None, errorCode=None, errorMessage=None, hasResultSet=None) 2024-04-04 06:58:22,578:DEBUG:impala.hiveserver2:_wait_to_finish: waited 0.0021505355834960938 seconds so far 2024-04-04 06:58:22,578:DEBUG:impala.hiveserver2:Query finished 2024-04-04 06:58:22,578:DEBUG:impala.hiveserver2:Executing query SHOW SCHEMAS 2024-04-04 06:58:22,579:DEBUG:impala.hiveserver2:_execute_async: self._buffer=Batch() self._description=None self._last_operation_active=True self._last_operation=TOperationHandle(operationId=THandleIdentifier(guid=b'\x08.\x1c\x0c\xfe\x1cEn\x00\x00\x00\x00\xa1\xbe%', secret=b'\x1b\xeb\xec\xab\x15\x85B\xec\xa6\xd1x3\xcf\xb1w/'), operationType=0, hasResultSet=False, modifiedRowCount=None)
2024-04-04 06:58:22,579:DEBUG:impala.hiveserver2:_reset_state: Resetting cursor state
2024-04-04 06:58:22,579:DEBUG:impala.hiveserver2:CloseOperation: req=TCloseOperationReq(operationHandle=TOperationHandle(operationId=THandleIdentifier(guid=b'\x08.\x1c\x0c\xfe\x1cEn\x00\x00\x00\x00\xa1`\xbe%', secret=b'\x1b\xeb\xec\xab\x15\x85B\xec\xa6\xd1x3\xcf\xb1w/'), operationType=0, hasResultSet=False, modifiedRowCount=None))
2024-04-04 06:58:22,579:DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=3)
2024-04-04 06:58:22,579:DEBUG:impala.hiveserver2:Transport opened
2024-04-04 06:58:22,581:DEBUG:impala.hiveserver2:CloseOperation: resp=TCloseOperationResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None))
2024-04-04 06:58:22,582:DEBUG:impala.hiveserver2:_execute_async: self._buffer=Batch() self._description=None self._last_operation_active=False self._last_operation=None
2024-04-04 06:58:22,582:DEBUG:impala.hiveserver2:ExecuteStatement: req=TExecuteStatementReq(sessionHandle=TSessionHandle(sessionId=THandleIdentifier(guid=b"\x01c\xbd\x84'\xedG^\xb3\xf1y\x13V\xf1\xa6\x91", secret=b'\x1b\xeb\xec\xab\x15\x85B\xec\xa6\xd1x3\xcf\xb1w/')), statement='SHOW SCHEMAS', confOverlay=None, runAsync=True)
2024-04-04 06:58:22,582:DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=3)
2024-04-04 06:58:22,582:DEBUG:impala.hiveserver2:Transport opened
pyroman30
commented
7 months ago If I change the cache parameters, it helps for a while, then it doesn't work again
Getting table list is not working while creating dataset and in SQL Lab, got following error:
There was an error loading the tablesHow to reproduce the bug
Go to 'Datasets'
Click on '+ Dataset'
Choose impala datasource
Choose schema
See error:
There was an error loading the tablesGo to 'SQL' -> 'SQL Lab'
Choose impala datasource
Choose schema
See error:
There was an error loading the tablesExpected results
List of tables are seen in at
SEE TABLE SCHEMAfieldActual results
Nothing happens and error message appear at UI
Screenshots
Environment
2.1.0Python 3.8.120.18.00.16.0Checklist
Make sure to follow these steps before submitting your issue - thank you!
Additional context
Here is logs that I found, but it contains no error:
Also I tested when schema has about 40 tables and 0 tables with same result