search

cloudera / hue

Open source SQL Query Assistant service for Databases/Warehouses
https://cloudera.com
Apache License 2.0
1.17k stars 366 forks source link

when beeswax use spark thrift instead of hiveserver2 ,error : Autocomplete data fetching error: list index out of range #364

Closed lemontreeshy closed 7 years ago

lemontreeshy commented 8 years ago

image

I started spark-sql thriftserver : /opt/spark/sbin/start-thriftserver.sh --hiveconf hive.server2.thrift.port=12000 --master yarn

[beeswax]

Host where HiveServer2 is running.

If Kerberos security is enabled, use fully-qualified domain name (FQDN).

hive_server_host=analysis

Port where HiveServer2 Thrift server runs on.

hive_server_port=12000

and I can run spark-sql in beeswax

romainr commented 8 years ago

There are some missing parts in the Spark SQL API that we are tracking in https://issues.cloudera.org/browse/HUE-2985

romainr commented 8 years ago

Which Spark version are you using? On 1.6.1 I can't even list the tables

lemontreeshy commented 8 years ago

I'm using spark 1.6.0

simonvanderveldt commented 3 years ago

@romainr Was anything fixed/changed for this and was that the reason you closed this issue? We're still seeing the same message in the Table browser image

Logs from Hue:

[16/Jul/2021 12:02:53 +0200] decorators   ERROR    Error running describe
Traceback (most recent call last):
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/decorators.py", line 119, in wrapper
    return f(*args, **kwargs)
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/api.py", line 1039, in describe
    describe = get_api(request, snippet).describe(notebook, snippet, database, table, column=column)
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/connectors/base.py", line 646, in describe
    describe_database = self.describe_database(notebook, snippet, database)
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/connectors/hiveserver2.py", line 950, in describe_database
    return db.get_database(database)
  File "/usr/share/hue/apps/beeswax/src/beeswax/server/dbms.py", line 346, in get_database
    return self.client.get_database(database)
  File "/usr/share/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 1510, in get_database
    return self._client.get_database(database)
  File "/usr/share/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 850, in get_database
    return HiveServerTRowSet(desc_results.results, desc_schema.schema).cols(cols)[0]  # Should only contain one row
  File "/usr/share/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 268, in cols
    cols = [rs.full_col(name) for name in col_names]
  File "/usr/share/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 268, in <listcomp>
    cols = [rs.full_col(name) for name in col_names]
  File "/usr/share/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 299, in full_col
    pos = self._get_col_position(colName)
  File "/usr/share/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 303, in _get_col_position
    return list(filter(lambda i_col1: i_col1[1].columnName == column_name, enumerate(self.schema.columns)))[0][0]
IndexError: list index out of range

We're using the Spark thriftserver running Spark 3.1.1 and Hue 4.10.0

romainr commented 3 years ago

https://github.com/cloudera/hue/blob/master/tools/docker/hue/Dockerfile#L61

simonvanderveldt commented 3 years ago

@romainr hmm, we're using that image/the upstream Hue image and still seeing this error. Should I create a new issue?

romainr commented 3 years ago

You can, but would recommend to use the SQL Alchemy interface and not Hue's HiveServer2 as it should work out of the box https://gethue.com/blog/querying-spark-sql-with-spark-thrift-server-and-hue-editor/