Compatibility Issues with Aliyun MaxCompute after Upgrading Superset to 4.0

Smallhi commented 4 months ago

Bug description

We are currently utilizing Aliyun MaxCompute as our database specification. However, after our recent upgrade to Superset version 4.0, we have encountered several issues.

Superset appears to be incompatible with tables formatted as 'sales.dwd_user_action'. The existing datasets cannot be reached through Superset. However, by manually modifying the database entry to 'dwd_user_action', the table becomes accessible.

When attempting to create a dataset for the 'dwd_user_action' table, which contains a large number of columns, Superset fails with the following error message: "The SQL is invalid and cannot be parsed. Unable to load columns for the selected table. Please select a different table."

Due to this being a production environment, I have temporarily made changes to the code in helpers.py, but this is not a sustainable solution.

We request assistance in resolving these compatibility issues. Your help would be greatly appreciated.

Thank you for your attention to this matter.

def get_query_str_extended(
        self,
        query_obj: QueryObjectDict,
        mutate: bool = True,
    ) -> QueryStringExtended:
        sqlaq = self.get_sqla_query(**query_obj)
        sql = self.database.compile_sqla_query(sqlaq.sqla_query)
        sql = self._apply_cte(sql, sqlaq.cte)
        try:
            if 'maxcompute' == self.db_engine_spec.engine:
                sql = sqlparse.format(sql, reindent=True)
            else:
                sql = SQLStatement(sql, engine=self.db_engine_spec.engine).format()
        except SupersetParseError:
            logger.warning("Unable to parse SQL to format it, passing it as-is"

How to reproduce the bug

pip install pyodps
use odps datasource
import dataset
we can't import table like 'sales.dwd_user_action' but 'dwd_user_action' is ok

Screenshots/recordings

No response

Superset version

4.0.0

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

Checklist

[ ] I have searched Superset docs and Slack and didn't find a solution to my problem.
[ ] I have searched the GitHub issue tracker and didn't find a similar bug report.
[ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

TimothyDing commented 3 months ago

@zhaoyongjie The submitter is a senior developer at Alibaba who uses Superset, and this PR is very important for promoting Superset to MaxCompute users.Please have a look.

zhaoyongjie commented 3 months ago

@TimothyDing Thanks for the mention, I'll check it out and reply you guys after finding any clues.

apache / superset