trinodb / trino-python-client

Python client for Trino
Apache License 2.0
327 stars 163 forks source link

Suppress failure when reading $partitions system table in get_indexes #426

Closed LittleWat closed 9 months ago

LittleWat commented 10 months ago

Description

Presto does support the $partitions table suffix per the release notes but Trino seems not to support this so this should be removed (?)

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/flask_appbuilder/api/__init__.py", line 110, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/views/base_api.py", line 127, in wraps
    raise ex
  File "/app/superset/views/base_api.py", line 121, in wraps
    duration, response = time_function(f, self, *args, **kwargs)
  File "/app/superset/utils/core.py", line 1454, in time_function
    response = func(*args, **kwargs)
  File "/app/superset/utils/log.py", line 255, in wrapper
    value = f(*args, **kwargs)
  File "/app/superset/databases/api.py", line 794, in table_extra_metadata
    payload = database.db_engine_spec.extra_table_metadata(
  File "/app/superset/db_engine_specs/trino.py", line 66, in extra_table_metadata
    if indexes := database.get_indexes(table_name, schema_name):
  File "/app/superset/models/core.py", line 863, in get_indexes
    return self.db_engine_spec.get_indexes(self, inspector, table_name, schema)
  File "/app/superset/db_engine_specs/base.py", line 1298, in get_indexes
    return inspector.get_indexes(table_name, schema)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/reflection.py", line 605, in get_indexes
    return self.dialect.get_indexes(
  File "/usr/local/lib/python3.9/site-packages/trino/sqlalchemy/dialect.py", line 283, in get_indexes
    partitioned_columns = self._get_columns(connection, f"{table_name}$partitions", schema, **kw)
  File "/usr/local/lib/python3.9/site-packages/trino/sqlalchemy/dialect.py", line 178, in _get_columns
    res = connection.execute(sql.text(query), {"schema": schema, "table": table_name})
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1306, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement
    ret = self._execute_context(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
    util.raise_(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
    raise exception
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.9/site-packages/trino/sqlalchemy/dialect.py", line 399, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.9/site-packages/trino/dbapi.py", line 587, in execute
    self._iterator = iter(self._query.execute())
  File "/usr/local/lib/python3.9/site-packages/trino/client.py", line 810, in execute
    self._result.rows += self.fetch()
  File "/usr/local/lib/python3.9/site-packages/trino/client.py", line 830, in fetch
    status = self._request.process(response)
  File "/usr/local/lib/python3.9/site-packages/trino/client.py", line 609, in process
    raise self._process_error(response["error"], response.get("id"))
sqlalchemy.exc.ProgrammingError: (trino.exceptions.TrinoUserError) TrinoUserError(type=USER_ERROR, name=NOT_SUPPORTED, message="Invalid Hudi table name (unknown type 'partitions'): my-table$partitions", query_id=20231129_142219_00171_dk2ne)
[SQL: SELECT
    "column_name",
    "data_type",
    "column_default",
    UPPER("is_nullable") AS "is_nullable"
FROM "information_schema"."columns"
WHERE "table_schema" = ?
  AND "table_name" = ?
ORDER BY "ordinal_position" ASC]
[parameters: ('my-schema', 'my-table$partitions')]

Non-technical explanation

Release notes

(x) This is not user-visible or docs only and no release notes are required. ( ) Release notes are required, please propose a release note for me. ( ) Release notes are required, with the following suggested text:

* Fix error when fetching Hudi tables schema. ({issue}`https://github.com/apache/superset/issues/21945`)
cla-bot[bot] commented 10 months ago

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot[bot] commented 9 months ago

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

LittleWat commented 9 months ago

I could confirm that this patch fixes the issue, thanks!! : スクリーンショット 2023-12-20 16 57 12 I'm glad if you could merge this so that we don't have to use the custom library. 🙏

LittleWat commented 9 months ago

thank you for your review in detail and sharing the doc! Fixed following your suggestion 🙇

hashhar commented 9 months ago

edited commit message to add some context, merging. Thanks @LittleWat.

LittleWat commented 9 months ago

thank you for updating the commit message and merge this! this helps our project!