[4.0.2] SQL LAB int argument error since version 4

fzhan commented 1 month ago

Bug description

Since upgraded to version 4, currently running 4.0.2, SQL LAB has stop working on all previous functioning database connections, mariadb, postgres and mssql.

How to reproduce the bug

Add a db connection,
Go to sql lab
Run a query

Here's the log:

2024-08-05 00:02:35,647:INFO:superset.sql_lab:Query 940: Storing results in results backend, key: 3f218da6-8f9f-47c1-a68f-8b1abfc89b12
SupersetErrorsException
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.10/site-packages/flask_appbuilder/security/decorators.py", line 95, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/views/base_api.py", line 127, in wraps
    raise ex
  File "/app/superset/views/base_api.py", line 121, in wraps
    duration, response = time_function(f, self, *args, **kwargs)
  File "/app/superset/utils/core.py", line 1470, in time_function
    response = func(*args, **kwargs)
  File "/app/superset/views/base_api.py", line 93, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/utils/log.py", line 255, in wrapper
    value = f(*args, **kwargs)
  File "/app/superset/sqllab/api.py", line 409, in execute_sql_query
    command_result: CommandResult = command.run()
  File "/app/superset/commands/sql_lab/execute.py", line 121, in run
    raise ex
  File "/app/superset/commands/sql_lab/execute.py", line 103, in run
    status = self._run_sql_json_exec_from_scratch()
  File "/app/superset/commands/sql_lab/execute.py", line 163, in _run_sql_json_exec_from_scratch
    raise ex
  File "/app/superset/commands/sql_lab/execute.py", line 158, in _run_sql_json_exec_from_scratch
    return self._sql_json_executor.execute(
  File "/app/superset/sqllab/sql_json_executer.py", line 111, in execute
    raise SupersetErrorsException(
superset.exceptions.SupersetErrorsException: [SupersetError(message="int() argument must be a string, a bytes-like object or a real number, not 'set'", error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'MySQL', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]

Screenshots/recordings

Superset version

4.0.2

Python version

3.10

Node version

16

Browser

Chrome

Additional context

No response

Checklist

[X] I have searched Superset docs and Slack and didn't find a solution to my problem.
[X] I have searched the GitHub issue tracker and didn't find a similar bug report.
[X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

dosubot[bot] commented 1 month ago

The issue with SQL Lab stopping working and the int() function receiving an argument of type 'set' after upgrading to version 4.0.2 is likely related to a bug introduced in the new version. Specifically, the changelog mentions several fixes related to SQL Lab and database interactions, such as:

#27646 fix: Leverage actual database for rendering Jinjarized SQL
#27636 fix(sqllab): unable to remove table
#27391 fix(sqllab): Close already removed tab
#27313 fix(sqllab): Missing empty query result state
#27295 fix(sqllab): invalid dump sql shown after closing tab
#27181 fix(sqllab): typeahead search is broken in db selector ^[1].

To resolve the error related to the int() function receiving an argument of type 'set', you should check if there are any updates or patches available that address this specific issue. If not, you may need to manually debug the code to identify where the int() function is being called with a 'set' type argument and correct it.

Here is a general approach to debug and fix the issue:

Identify the part of the code where the int() function is being called with a 'set' type argument.
Ensure that the argument passed to the int() function is of a type that can be converted to an integer, such as a string or a number.

For example, if you find code like this:

value = some_set
result = int(value)

You should change it to:

value = some_set
if isinstance(value, set):
    value = list(value)[0]  # or some other logic to extract a single value
result = int(value)

Additionally, there are configuration changes and migration steps required when upgrading to version 4.0.2 that could impact SQL Lab functionality. Specifically:

Deprecation of Endpoints:
- The endpoints /superset/sqllab, /superset/sqllab/history, and /sqllab/my_queries have been removed. You should use /sqllab, /sqllab/history, and /savedqueryview/list/?_flt_0_user={get_user_id()} instead ^[2].
Deprecation of KV_STORE Feature Flag:
- The KV_STORE feature flag and its related assets such as the API endpoint and keyvalue table have been deprecated. This affects the SHARE_QUERIES_VIA_KV_STORE feature flag, which allows sharing SQL Lab queries without saving the query ^[2].
Index Addition:
- An index has been added to query.sql_editor_id to improve performance, which may cause downtime on large deployments ^[2].
Database Column Updates:
- Some database columns have been updated to MediumText, which may require migration steps ^[2].
Removal of Deprecated Redirect Endpoint:
- The deprecated redirect endpoint has been removed, which may affect existing configurations ^[2].

Review the CHANGELOG.md and UPDATING.md files, back up your databases, and adjust your feature flags and configurations accordingly.

To continue talking to Dosu, mention @dosu.

rusackas commented 1 month ago

Hmm, something seems funny here... if you couldn't run any query in postgres/mysql, there'd be pitchforks and torches out on the streets by now. Can you give an example failing query, preferably using an example dataset?

fzhan commented 1 month ago

@rusackas I do have some custom libs:

pip install sqlalchemy-bigquery redis pymssql pyodbc pymysql mysql-connector-python holidays==0.32 prophet==1.1.4 authlib celery==5.3.6 kombu==5.4.0rc3 sqlglot

apache / superset