apache / superset

Apache Superset is a Data Visualization and Data Exploration Platform
https://superset.apache.org/
Apache License 2.0
60.52k stars 13.07k forks source link

Database Issue when using mariadb 10.4, illegal mixx #29483

Open cosmincioranu opened 4 days ago

cosmincioranu commented 4 days ago

Bug description

Issueing a database connection triggers An error occurred while fetching databases: (MySQLdb.OperationalError) (1267, "Illegal mix of collations (utf8_bin,NONE) and (binary,IGNORABLE) for operation '='") [SQL: SELECT count(*) AS count_1 FROM (SELECT dbs.uuid AS dbs_uuid, dbs.created_on AS dbs_created_on, dbs.changed_on AS dbs_changed_on, dbs.id AS dbs_id, dbs.verbose_name AS dbs_verbose_name, dbs.database_name AS dbs_database_name, dbs.sqlalchemy_uri AS dbs_sqlalchemy_uri, dbs.password AS dbs_password, dbs.cache_timeout AS dbs_cache_timeout, dbs.select_as_create_table_as AS dbs_select_as_create_table_as, dbs.expose_in_sqllab AS dbs_expose_in_sqllab, dbs.configuration_method AS dbs_configuration_method, dbs.allow_run_async AS dbs_allow_run_async, dbs.allow_file_upload AS dbs_allow_file_upload, dbs.allow_ctas AS dbs_allow_ctas, dbs.allow_cvas AS dbs_allow_cvas, dbs.allow_dml AS dbs_allow_dml, dbs.force_ctas_schema AS dbs_force_ctas_schema, dbs.extra AS dbs_extra, dbs.encrypted_extra AS dbs_encrypted_extra, dbs.impersonate_user AS dbs_impersonate_user, dbs.server_cert AS dbs_server_cert, dbs.is_managed_externally AS dbs_is_managed_externally, dbs.external_url AS dbs_external_url, dbs.created_by_fk AS dbs_created_by_fk, dbs.changed_by_fk AS dbs_changed_by_fk, changed_by.id AS changed_by_id, changed_by.first_name AS changed_by_first_name, changed_by.last_name AS changed_by_last_name, changed_by.username AS changed_by_username, changed_by.password AS changed_by_password, changed_by.active AS changed_by_active, changed_by.email AS changed_by_email, changed_by.last_login AS changed_by_last_login, changed_by.login_count AS changed_by_login_count, changed_by.fail_login_count AS changed_by_fail_login_count, changed_by.created_on AS changed_by_created_on, changed_by.changed_on AS changed_by_changed_on, changed_by.created_by_fk AS changed_by_created_by_fk, changed_by.changed_by_fk AS changed_by_changed_by_fk, created_by.id AS created_by_id, created_by.first_name AS created_by_first_name, created_by.last_name AS created_by_last_name, created_by.username AS created_by_username, created_by.password AS created_by_password, created_by.active AS created_by_active, created_by.email AS created_by_email, created_by.last_login AS created_by_last_login, created_by.login_count AS created_by_login_count, created_by.fail_login_count AS created_by_fail_login_count, created_by.created_on AS created_by_created_on, created_by.changed_on AS created_by_changed_on, created_by.created_by_fk AS created_by_created_by_fk, created_by.changed_by_fk AS created_by_changed_by_fk FROM dbs LEFT OUTER JOIN ab_user AS changed_by ON dbs.changed_by_fk = changed_by.id LEFT OUTER JOIN ab_user AS created_by ON dbs.created_by_fk = created_by.id WHERE concat(concat(concat(concat(%s, dbs.database_name), %s), CAST(dbs.id AS CHAR)), %s) IN (NULL) AND (1 != 1) OR dbs.database_name IN (NULL) AND (1 != 1)) AS anon_1] [parameters: ('[', '].(id:', ')')] (Background on this error at: https://sqlalche.me/e/14/e3q8)

How to reproduce the bug

  1. goto to database connection of a newly created user.

Screenshots/recordings

image

Superset version

master / latest-dev

Python version

3.9

Node version

16

Browser

Chrome

Additional context

Jul 4 10:32:24 dssuperset python3.9[17740]: 2024-07-04 10:32:24,754:ERROR:superset.views.base:(MySQLdb.OperationalError) (1267, "Illegal mix of collations (utf8_bin,NONE) and (binary,IGNORABLE) for operation '='") Jul 4 10:32:24 dssuperset python3.9[17740]: [SQL: SELECT count() AS count_1 Jul 4 10:32:24 dssuperset python3.9[17740]: FROM (SELECT dbs.uuid AS dbs_uuid, dbs.created_on AS dbs_created_on, dbs.changed_on AS dbs_changed_on, dbs.id AS dbs_id, dbs.verbose_name AS dbs_verbose_name, dbs.database_name AS dbs_database_name, dbs.sqlalchemy_uri AS dbs_sqlalchemy_uri, dbs.password AS dbs_password, dbs.cache_timeout AS dbs_cache_timeout, dbs.select_as_create_table_as AS dbs_select_as_create_table_as, dbs.expose_in_sqllab AS dbs_expose_in_sqllab, dbs.configuration_method AS dbs_configuration_method, dbs.allow_run_async AS dbs_allow_run_async, dbs.allow_file_upload AS dbs_allow_file_upload, dbs.allow_ctas AS dbs_allow_ctas, dbs.allow_cvas AS dbs_allow_cvas, dbs.allow_dml AS dbs_allow_dml, dbs.force_ctas_schema AS dbs_force_ctas_schema, dbs.extra AS dbs_extra, dbs.encrypted_extra AS dbs_encrypted_extra, dbs.impersonate_user AS dbs_impersonate_user, dbs.server_cert AS dbs_server_cert, dbs.is_managed_externally AS dbs_is_managed_externally, dbs.external_url AS dbs_external_url, dbs.created_by_fk AS dbs_created_by_fk, dbs.changed_by_fk AS dbs_changed_by_fk, changed_by.id AS changed_by_id, changed_by.first_name AS changed_by_first_name, changed_by.last_name AS changed_by_last_name, changed_by.username AS changed_by_username, changed_by.password AS changed_by_password, changed_by.active AS changed_by_active, changed_by.email AS changed_by_email, changed_by.last_login AS changed_by_last_login, changed_by.login_count AS changed_by_login_count, changed_by.fail_login_count AS changed_by_fail_login_count, changed_by.created_on AS changed_by_created_on, changed_by.changed_on AS changed_by_changed_on, changed_by.created_by_fk AS changed_by_created_by_fk, changed_by.changed_by_fk AS changed_by_changed_by_fk, created_by.id AS created_by_id, created_by.first_name AS created_by_first_name, created_by.last_name AS created_by_last_name, created_by.username AS created_by_username, created_by.password AS created_by_password, created_by.active AS created_by_active, created_by.email AS created_by_email, created_by.last_login AS created_by_last_login, created_by.login_count AS created_by_login_count, created_by.fail_login_count AS created_by_fail_login_count, created_by.created_on AS created_by_created_on, created_by.changed_on AS created_by_changed_on, created_by.created_by_fk AS created_by_created_by_fk, created_by.changed_by_fk AS created_by_changed_by_fk Jul 4 10:32:24 dssuperset python3.9[17740]: FROM dbs LEFT OUTER JOIN ab_user AS changed_by ON dbs.changed_by_fk = changed_by.id LEFT OUTER JOIN ab_user AS created_by ON dbs.created_by_fk = created_by.id Jul 4 10:32:24 dssuperset python3.9[17740]: WHERE dbs.allow_file_upload = 1 AND true = 1 AND (concat(concat(concat(concat(%s, dbs.database_name), %s), CAST(dbs.id AS CHAR)), %s) IN (NULL) AND (1 != 1) OR dbs.database_name IN (NULL) AND (1 != 1))) AS anon_1] Jul 4 10:32:24 dssuperset python3.9[17740]: [parameters: ('[', '].(id:', ')')] Jul 4 10:32:24 dssuperset python3.9[17740]: (Background on this error at: https://sqlalche.me/e/14/e3q8) Jul 4 10:32:24 dssuperset python3.9[17740]: Traceback (most recent call last): Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context Jul 4 10:32:24 dssuperset python3.9[17740]: self.dialect.do_execute( Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute Jul 4 10:32:24 dssuperset python3.9[17740]: cursor.execute(statement, parameters) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/MySQLdb/cursors.py", line 179, in execute Jul 4 10:32:24 dssuperset python3.9[17740]: res = self._query(mogrified_query) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/MySQLdb/cursors.py", line 330, in _query Jul 4 10:32:24 dssuperset python3.9[17740]: db.query(q) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/MySQLdb/connections.py", line 261, in query Jul 4 10:32:24 dssuperset python3.9[17740]: _mysql.connection.query(self, query) Jul 4 10:32:24 dssuperset python3.9[17740]: MySQLdb.OperationalError: (1267, "Illegal mix of collations (utf8_bin,NONE) and (binary,IGNORABLE) for operation '='") Jul 4 10:32:24 dssuperset python3.9[17740]: The above exception was the direct cause of the following exception: Jul 4 10:32:24 dssuperset python3.9[17740]: Traceback (most recent call last): Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/superset/views/base.py", line 252, in wraps Jul 4 10:32:24 dssuperset python3.9[17740]: return f(self, args, kwargs) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/superset/views/base_api.py", line 487, in get_list_headless Jul 4 10:32:24 dssuperset python3.9[17740]: duration, response = time_function(super().get_list_headless, kwargs) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/superset/utils/core.py", line 1470, in time_function Jul 4 10:32:24 dssuperset python3.9[17740]: response = func(*args, *kwargs) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/flask_appbuilder/api/init.py", line 1619, in get_list_headless Jul 4 10:32:24 dssuperset python3.9[17740]: count, lst = self.datamodel.query( Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/flask_appbuilder/models/sqla/interface.py", line 489, in query Jul 4 10:32:24 dssuperset python3.9[17740]: count = self.query_count(query, filters, select_columns) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/flask_appbuilder/models/sqla/interface.py", line 396, in query_count Jul 4 10:32:24 dssuperset python3.9[17740]: return self._apply_inner_all( Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/orm/query.py", line 3176, in count Jul 4 10:32:24 dssuperset python3.9[17740]: return self._from_self(col).enable_eagerloads(False).scalar() Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/orm/query.py", line 2893, in scalar Jul 4 10:32:24 dssuperset python3.9[17740]: ret = self.one() Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/orm/query.py", line 2870, in one Jul 4 10:32:24 dssuperset python3.9[17740]: return self._iter().one() Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/orm/query.py", line 2916, in _iter Jul 4 10:32:24 dssuperset python3.9[17740]: result = self.session.execute( Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/orm/session.py", line 1717, in execute Jul 4 10:32:24 dssuperset python3.9[17740]: result = conn._execute_20(statement, params or {}, execution_options) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20 Jul 4 10:32:24 dssuperset python3.9[17740]: return meth(self, args_10style, kwargs_10style, execution_options) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection Jul 4 10:32:24 dssuperset python3.9[17740]: return connection._execute_clauseelement( Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement Jul 4 10:32:24 dssuperset python3.9[17740]: ret = self._execute_context( Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context Jul 4 10:32:24 dssuperset python3.9[17740]: self._handle_dbapi_exception( Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 2134, in _handle_dbapiexception Jul 4 10:32:24 dssuperset python3.9[17740]: util.raise( Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ Jul 4 10:32:24 dssuperset python3.9[17740]: raise exception Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context Jul 4 10:32:24 dssuperset python3.9[17740]: self.dialect.do_execute( Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute Jul 4 10:32:24 dssuperset python3.9[17740]: cursor.execute(statement, parameters) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/MySQLdb/cursors.py", line 179, in execute Jul 4 10:32:24 dssuperset python3.9[17740]: res = self._query(mogrified_query) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/MySQLdb/cursors.py", line 330, in _query Jul 4 10:32:24 dssuperset python3.9[17740]: db.query(q) Jul 4 10:32:24 dssuperset python3.9[17740]: File "/opt/superset/lib64/python3.9/site-packages/MySQLdb/connections.py", line 261, in query Jul 4 10:32:24 dssuperset python3.9[17740]: _mysql.connection.query(self, query) Jul 4 10:32:24 dssuperset python3.9[17740]: sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (1267, "Illegal mix of collations (utf8_bin,NONE) and (binary,IGNORABLE) for operation '='") Jul 4 10:32:24 dssuperset python3.9[17740]: [SQL: SELECT count() AS count_1 Jul 4 10:32:24 dssuperset python3.9[17740]: FROM (SELECT dbs.uuid AS dbs_uuid, dbs.created_on AS dbs_created_on, dbs.changed_on AS dbs_changed_on, dbs.id AS dbs_id, dbs.verbose_name AS dbs_verbose_name, dbs.database_name AS dbs_database_name, dbs.sqlalchemy_uri AS dbs_sqlalchemy_uri, dbs.password AS dbs_password, dbs.cache_timeout AS dbs_cache_timeout, dbs.select_as_create_table_as AS dbs_select_as_create_table_as, dbs.expose_in_sqllab AS dbs_expose_in_sqllab, dbs.configuration_method AS dbs_configuration_method, dbs.allow_run_async AS dbs_allow_run_async, dbs.allow_file_upload AS dbs_allow_file_upload, dbs.allow_ctas AS dbs_allow_ctas, dbs.allow_cvas AS dbs_allow_cvas, dbs.allow_dml AS dbs_allow_dml, dbs.force_ctas_schema AS dbs_force_ctas_schema, dbs.extra AS dbs_extra, dbs.encrypted_extra AS dbs_encrypted_extra, dbs.impersonate_user AS dbs_impersonate_user, dbs.server_cert AS dbs_server_cert, dbs.is_managed_externally AS dbs_is_managed_externally, dbs.external_url AS dbs_external_url, dbs.created_by_fk AS dbs_created_by_fk, dbs.changed_by_fk AS dbs_changed_by_fk, changed_by.id AS changed_by_id, changed_by.first_name AS changed_by_first_name, changed_by.last_name AS changed_by_last_name, changed_by.username AS changed_by_username, changed_by.password AS changed_by_password, changed_by.active AS changed_by_active, changed_by.email AS changed_by_email, changed_by.last_login AS changed_by_last_login, changed_by.login_count AS changed_by_login_count, changed_by.fail_login_count AS changed_by_fail_login_count, changed_by.created_on AS changed_by_created_on, changed_by.changed_on AS changed_by_changed_on, changed_by.created_by_fk AS changed_by_created_by_fk, changed_by.changed_by_fk AS changed_by_changed_by_fk, created_by.id AS created_by_id, created_by.first_name AS created_by_first_name, created_by.last_name AS created_by_last_name, created_by.username AS created_by_username, created_by.password AS created_by_password, created_by.active AS created_by_active, created_by.email AS created_by_email, created_by.last_login AS created_by_last_login, created_by.login_count AS created_by_login_count, created_by.fail_login_count AS created_by_fail_login_count, created_by.created_on AS created_by_created_on, created_by.changed_on AS created_by_changed_on, created_by.created_by_fk AS created_by_created_by_fk, created_by.changed_by_fk AS created_by_changed_by_fk Jul 4 10:32:24 dssuperset python3.9[17740]: FROM dbs LEFT OUTER JOIN ab_user AS changed_by ON dbs.changed_by_fk = changed_by.id LEFT OUTER JOIN ab_user AS created_by ON dbs.created_by_fk = created_by.id Jul 4 10:32:24 dssuperset python3.9[17740]: WHERE dbs.allow_file_upload = 1 AND true = 1 AND (concat(concat(concat(concat(%s, dbs.database_name), %s), CAST(dbs.id AS CHAR)), %s) IN (NULL) AND (1 != 1) OR dbs.database_name IN (NULL) AND (1 != 1))) AS anon_1] Jul 4 10:32:24 dssuperset python3.9[17740]: [parameters: ('[', '].(id:', ')')] Jul 4 10:32:24 dssuperset python3.9[17740]: (Background on this error at: https://sqlalche.me/e/14/e3q8) Jul 4 10:32:25 dssuperset python3.9[17739]: 2024-07-04 10:32:25,588:INFO:flask_wtf.csrf:The CSRF token is missing. Jul 4 10:32:25 dssuperset python3.9[17739]: Refresh CSRF token error Jul 4 10:32:25 dssuperset python3.9[17739]: Traceback (most recent call last): Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask_wtf/csrf.py", line 261, in protect Jul 4 10:32:25 dssuperset python3.9[17739]: validate_csrf(self._get_csrf_token()) Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask_wtf/csrf.py", line 100, in validate_csrf Jul 4 10:32:25 dssuperset python3.9[17739]: raise ValidationError("The CSRF token is missing.") Jul 4 10:32:25 dssuperset python3.9[17739]: wtforms.validators.ValidationError: The CSRF token is missing. Jul 4 10:32:25 dssuperset python3.9[17739]: During handling of the above exception, another exception occurred: Jul 4 10:32:25 dssuperset python3.9[17739]: Traceback (most recent call last): Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask/app.py", line 1482, in full_dispatch_request Jul 4 10:32:25 dssuperset python3.9[17739]: rv = self.preprocess_request() Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask/app.py", line 1974, in preprocess_request Jul 4 10:32:25 dssuperset python3.9[17739]: rv = self.ensure_sync(before_func)() Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask_wtf/csrf.py", line 229, in csrf_protect Jul 4 10:32:25 dssuperset python3.9[17739]: self.protect() Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask_wtf/csrf.py", line 264, in protect Jul 4 10:32:25 dssuperset python3.9[17739]: self._error_response(e.args[0]) Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask_wtf/csrf.py", line 307, in _error_response Jul 4 10:32:25 dssuperset python3.9[17739]: raise CSRFError(reason) Jul 4 10:32:25 dssuperset python3.9[17739]: flask_wtf.csrf.CSRFError: 400 Bad Request: The CSRF token is missing. Jul 4 10:32:25 dssuperset python3.9[17739]: 2024-07-04 10:32:25,589:WARNING:superset.views.base:Refresh CSRF token error Jul 4 10:32:25 dssuperset python3.9[17739]: Traceback (most recent call last): Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask_wtf/csrf.py", line 261, in protect Jul 4 10:32:25 dssuperset python3.9[17739]: validate_csrf(self._get_csrf_token()) Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask_wtf/csrf.py", line 100, in validate_csrf Jul 4 10:32:25 dssuperset python3.9[17739]: raise ValidationError("The CSRF token is missing.") Jul 4 10:32:25 dssuperset python3.9[17739]: wtforms.validators.ValidationError: The CSRF token is missing. Jul 4 10:32:25 dssuperset python3.9[17739]: During handling of the above exception, another exception occurred: Jul 4 10:32:25 dssuperset python3.9[17739]: Traceback (most recent call last): Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask/app.py", line 1482, in full_dispatch_request Jul 4 10:32:25 dssuperset python3.9[17739]: rv = self.preprocess_request() Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask/app.py", line 1974, in preprocess_request Jul 4 10:32:25 dssuperset python3.9[17739]: rv = self.ensure_sync(before_func)() Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask_wtf/csrf.py", line 229, in csrf_protect Jul 4 10:32:25 dssuperset python3.9[17739]: self.protect() Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask_wtf/csrf.py", line 264, in protect Jul 4 10:32:25 dssuperset python3.9[17739]: self._error_response(e.args[0]) Jul 4 10:32:25 dssuperset python3.9[17739]: File "/opt/superset/lib64/python3.9/site-packages/flask_wtf/csrf.py", line 307, in _error_response Jul 4 10:32:25 dssuperset python3.9[17739]: raise CSRFError(reason) Jul 4 10:32:25 dssuperset python3.9[17739]: flask_wtf.csrf.CSRFError: 400 Bad Request: The CSRF token is missing.

Checklist

dosubot[bot] commented 4 days ago

Hey there, @cosmincioranu! πŸ‘‹ I'm here to help you with any bugs, questions, or contributions you have in mind. Let's squash those bugs together!

To resolve the "Illegal mix of collations" error when issuing a database connection using MariaDB 10.4 in Superset, you need to ensure that the collations of the database and the connection are compatible. This can be done by setting the collation explicitly in the connection parameters.

Here is an example of how you can modify the connection parameters to include the collation setting:

class ValidateDatabaseParametersCommand(BaseCommand):
    def __init__(self, properties: dict[str, Any]):
        self._properties = properties.copy()
        self._model: Optional[Database] = None

    def run(self) -> None:
        self.validate()

        engine = self._properties["engine"]
        driver = self._properties.get("driver")

        if engine in BYPASS_VALIDATION_ENGINES:
            return

        engine_spec = get_engine_spec(engine, driver)
        if not hasattr(engine_spec, "parameters_schema"):
            raise InvalidEngineError(
                SupersetError(
                    message=__(
                        'Engine "%(engine)s" cannot be configured through parameters.',
                        engine=engine,
                    ),
                    error_type=SupersetErrorType.GENERIC_DB_ENGINE_ERROR,
                    level=ErrorLevel.ERROR,
                ),
            )

        errors = engine_spec.validate_parameters(self._properties)  # type: ignore
        if errors:
            event_logger.log_with_context(action="validation_error", engine=engine)
            raise InvalidParametersError(errors)

        serialized_encrypted_extra = self._properties.get(
            "masked_encrypted_extra",
            "{}",
        )
        if self._model:
            serialized_encrypted_extra = engine_spec.unmask_encrypted_extra(
                self._model.encrypted_extra,
                serialized_encrypted_extra,
            )
        try:
            encrypted_extra = json.loads(serialized_encrypted_extra)
        except json.JSONDecodeError:
            encrypted_extra = {}

        if "parameters" not in self._properties:
            self._properties["parameters"] = {}
        self._properties["parameters"]["charset"] = "utf8mb4"
        self._properties["parameters"]["collation"] = "utf8mb4_unicode_ci"

        sqlalchemy_uri = engine_spec.build_sqlalchemy_uri(  # type: ignore
            self._properties.get("parameters"),
            encrypted_extra,
        )
        if self._model and sqlalchemy_uri == self._model.safe_sqlalchemy_uri():
            sqlalchemy_uri = self._model.sqlalchemy_uri_decrypted
        database = DatabaseDAO.build_db_for_connection_test(
            server_cert=self._properties.get("server_cert", ""),
            extra=self._properties.get("extra", "{}"),
            impersonate_user=self._properties.get("impersonate_user", False),
            encrypted_extra=serialized_encrypted_extra,
        )
        database.set_sqlalchemy_uri(sqlalchemy_uri)
        database.db_engine_spec.mutate_db_for_connection_test(database)

        alive = False
        with database.get_sqla_engine() as engine:
            try:
                with closing(engine.raw_connection()) as conn:
                    alive = engine.dialect.do_ping(conn)
            except Exception as ex:
                url = make_url_safe(sqlalchemy_uri)
                context = {
                    "hostname": url.host,
                    "password": url.password,
                    "port": url.port,
                    "username": url.username,
                    "database": url.database,
                }
                errors = database.db_engine_spec.extract_errors(ex, context)
                raise DatabaseTestConnectionFailedError(errors) from ex

        if not alive:
            raise DatabaseOfflineError(
                SupersetError(
                    message=__("Database is offline."),
                    error_type=SupersetErrorType.GENERIC_DB_ENGINE_ERROR,
                    level=ErrorLevel.ERROR,
                ),
            )

    def validate(self) -> None:
        if (database_id := self._properties.get("id")) is not None:
            self._model = DatabaseDAO.find_by_id(database_id)

In this example, the charset and collation parameters are added to the connection parameters to ensure that the connection uses the correct collation. Adjust the collation settings as needed to match your database configuration [1].

To continue talking to Dosu, mention @dosu.

cosmincioranu commented 4 days ago

Thank you for reply,

In superset_config.py i use

SQLALCHEMY_DATABASE_URI =
@./?collation=utf8_unicode_ci&charset=utf8'

which matches with the database used. Unfortunately this does not work.

-- Cosmin CIORANU Head of IT Department The Executive Agency For Higher Education, Research, Development And Innovation Funding (UEFISCDI) E-mail: @.*** Address: Mendeleev st., no. 21 – 25, room 15, sector 1, Bucharest, Romania, postal code 010362.

On Thu, 04 Jul 2024 11:36:27 +0300, dosubot[bot]
@.***> wrote:

Hey there, @cosmincioranu! πŸ‘‹ I'm here to help you with any bugs,
questions, or contributions you have in mind. Let's squash >those bugs
together!

To resolve the "Illegal mix of collations" error when issuing a database
connection using MariaDB 10.4 in Superset, you need >to ensure that the
collations of the database and the connection are compatible. This can
be done by setting the collation >explicitly in the connection
parameters.

Here is an example of how you can modify the connection parameters to
include the collation setting: class ValidateDatabaseParametersCommand(BaseCommand): def init(self, properties: dict[str, Any]): self._properties = properties.copy() self._model: Optional[Database] = None

def run(self) -> None: self.validate()

   engine = self._properties["engine"]
   driver = self._properties.get("driver")

   if engine in BYPASS_VALIDATION_ENGINES:
       return

   engine_spec = get_engine_spec(engine, driver)
   if not hasattr(engine_spec, "parameters_schema"):
       raise InvalidEngineError(
           SupersetError(
               message=__(
                   'Engine "%(engine)s" cannot be configured through  

parameters.', engine=engine, ), error_type=SupersetErrorType.GENERIC_DB_ENGINE_ERROR, level=ErrorLevel.ERROR, ), )

   errors = engine_spec.validate_parameters(self._properties)  #  

type: ignore if errors: event_logger.log_with_context(action="validation_error",
engine=engine) raise InvalidParametersError(errors)

   serialized_encrypted_extra = self._properties.get(
       "masked_encrypted_extra",
       "{}",
   )
   if self._model:
       serialized_encrypted_extra =  

engine_spec.unmask_encrypted_extra( self._model.encrypted_extra, serialized_encrypted_extra, ) try: encrypted_extra = json.loads(serialized_encrypted_extra) except json.JSONDecodeError: encrypted_extra = {}

   if "parameters" not in self._properties:
       self._properties["parameters"] = {}
   self._properties["parameters"]["charset"] = "utf8mb4"
   self._properties["parameters"]["collation"] = "utf8mb4_unicode_ci"

   sqlalchemy_uri = engine_spec.build_sqlalchemy_uri(  # type: ignore
       self._properties.get("parameters"),
       encrypted_extra,
   )
   if self._model and sqlalchemy_uri ==  

self._model.safe_sqlalchemy_uri(): sqlalchemy_uri = self._model.sqlalchemy_uri_decrypted database = DatabaseDAO.build_db_for_connection_test( server_cert=self._properties.get("server_cert", ""), extra=self._properties.get("extra", "{}"), impersonate_user=self._properties.get("impersonate_user",
False), encrypted_extra=serialized_encrypted_extra, ) database.set_sqlalchemy_uri(sqlalchemy_uri) database.db_engine_spec.mutate_db_for_connection_test(database)

   alive = False
   with database.get_sqla_engine() as engine:
       try:
           with closing(engine.raw_connection()) as conn:
               alive = engine.dialect.do_ping(conn)
       except Exception as ex:
           url = make_url_safe(sqlalchemy_uri)
           context = {
               "hostname": url.host,
               "password": url.password,
               "port": url.port,
               "username": url.username,
               "database": url.database,
           }
           errors = database.db_engine_spec.extract_errors(ex,  

context) raise DatabaseTestConnectionFailedError(errors) from ex

   if not alive:
       raise DatabaseOfflineError(
           SupersetError(
               message=__("Database is offline."),
               error_type=SupersetErrorType.GENERIC_DB_ENGINE_ERROR,
               level=ErrorLevel.ERROR,
           ),
       )

def validate(self) -> None: if (database_id := self._properties.get("id")) is not None: self._model = DatabaseDAO.find_by_id(database_id)

In this example, the charset and collation parameters are added to the
connection parameters to ensure that the >connection uses the correct
collation. Adjust the collation settings as needed to match your
database configuration [1].

To continue talking to Dosu, mention @dosu.

β€” Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

dosubot[bot] commented 4 days ago

Ensure that the collation and charset settings in your superset_config.py match the database settings. If the issue persists, try explicitly setting the collation and charset in the connection parameters within the Superset codebase.

Here's an example of how to modify the connection parameters:

class ValidateDatabaseParametersCommand(BaseCommand):
    def __init__(self, properties: dict[str, Any]):
        self._properties = properties.copy()
        self._model: Optional[Database] = None

    def run(self) -> None:
        self.validate()

        engine = self._properties["engine"]
        driver = self._properties.get("driver")

        if engine in BYPASS_VALIDATION_ENGINES:
            return

        engine_spec = get_engine_spec(engine, driver)
        if not hasattr(engine_spec, "parameters_schema"):
            raise InvalidEngineError(
                SupersetError(
                    message=__(
                        'Engine "%(engine)s" cannot be configured through parameters.',
                        engine=engine,
                    ),
                    error_type=SupersetErrorType.GENERIC_DB_ENGINE_ERROR,
                    level=ErrorLevel.ERROR,
                ),
            )

        errors = engine_spec.validate_parameters(self._properties)  # type: ignore
        if errors:
            event_logger.log_with_context(action="validation_error", engine=engine)
            raise InvalidParametersError(errors)

        serialized_encrypted_extra = self._properties.get(
            "masked_encrypted_extra",
            "{}",
        )
        if self._model:
            serialized_encrypted_extra = engine_spec.unmask_encrypted_extra(
                self._model.encrypted_extra,
                serialized_encrypted_extra,
            )
        try:
            encrypted_extra = json.loads(serialized_encrypted_extra)
        except json.JSONDecodeError:
            encrypted_extra = {}

        if "parameters" not in self._properties:
            self._properties["parameters"] = {}
        self._properties["parameters"]["charset"] = "utf8mb4"
        self._properties["parameters"]["collation"] = "utf8mb4_unicode_ci"

        sqlalchemy_uri = engine_spec.build_sqlalchemy_uri(  # type: ignore
            self._properties.get("parameters"),
            encrypted_extra,
        )
        if self._model and sqlalchemy_uri == self._model.safe_sqlalchemy_uri():
            sqlalchemy_uri = self._model.sqlalchemy_uri_decrypted
        database = DatabaseDAO.build_db_for_connection_test(
            server_cert=self._properties.get("server_cert", ""),
            extra=self._properties.get("extra", "{}"),
            impersonate_user=self._properties.get("impersonate_user", False),
            encrypted_extra=serialized_encrypted_extra,
        )
        database.set_sqlalchemy_uri(sqlalchemy_uri)
        database.db_engine_spec.mutate_db_for_connection_test(database)

        alive = False
        with database.get_sqla_engine() as engine:
            try:
                with closing(engine.raw_connection()) as conn:
                    alive = engine.dialect.do_ping(conn)
            except Exception as ex:
                url = make_url_safe(sqlalchemy_uri)
                context = {
                    "hostname": url.host,
                    "password": url.password,
                    "port": url.port,
                    "username": url.username,
                    "database": url.database,
                }
                errors = database.db_engine_spec.extract_errors(ex, context)
                raise DatabaseTestConnectionFailedError(errors) from ex

        if not alive:
            raise DatabaseOfflineError(
                SupersetError(
                    message=__("Database is offline."),
                    error_type=SupersetErrorType.GENERIC_DB_ENGINE_ERROR,
                    level=ErrorLevel.ERROR,
                ),
            )

    def validate(self) -> None:
        if (database_id := self._properties.get("id")) is not None:
            self._model = DatabaseDAO.find_by_id(database_id)

In this example, the charset and collation parameters are explicitly set to utf8mb4 and utf8mb4_unicode_ci, respectively, to ensure compatibility [1].

To continue talking to Dosu, mention @dosu.