Python error on macos caching_sha2_password

[KalitaAlexey]

I execute: docker-compose -p node-template -f docker-compose.substrate-node-template.yml up --build

I get: harvester-worker_1 | [2021-01-22 07:16:54,965: ERROR/ForkPoolWorker-2] Task app.tasks.accumulate_block_recursive[36094758-9c0e-4730-a370-b61ee11a8c42] raised unexpected: NotSupportedError("(mysql.connector.errors.NotSupportedError) Authentication plugin 'caching_sha2_password' is not supported",)

[arjanz]

Just to be sure this is not an issue with the newest version of MySQL I pinned the version to 8.0.23 in the docker-compose files, which is the version I always run my tests on. Do you still have the issue?

[KalitaAlexey]

Yes. I still see it.

[KalitaAlexey]

The same issue on Windows: sqlalchemy.exc.NotSupportedError: (mysql.connector.errors.NotSupportedError) Authentication plugin 'caching_sha2_password' is not supported

This time I checked initialization logs:

harvester-api_1      | Wait for database...
harvester-monitor_1  | Error:
harvester-monitor_1  | Unable to load celery application.
harvester-monitor_1  | The module scalecodec was not found.
...
mysql_1              | 2021-01-29T07:25:25.762178Z 6 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
...
mysql_1              | 2021-01-29 07:25:36+00:00 [Note] [Entrypoint]: Stopping temporary server
mysql_1              | 2021-01-29T07:25:36.415357Z 11 [System] [MY-013172] [Server] Received SHUTDOWN from user root. Shutting down mysqld (Version: 8.0.23).
mysql_1              | 2021-01-29T07:25:38.397633Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.23)  MySQL Community Server - GPL.
mysql_1              | 2021-01-29 07:25:38+00:00 [Note] [Entrypoint]: Temporary server stopped
mysql_1              |
mysql_1              | 2021-01-29 07:25:38+00:00 [Note] [Entrypoint]: MySQL init process done. Ready for start up.
mysql_1              |
mysql_1              | 2021-01-29T07:25:38.663364Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.23) starting as process 1
mysql_1              | 2021-01-29T07:25:38.675602Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
mysql_1              | 2021-01-29T07:25:38.874921Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
mysql_1              | 2021-01-29T07:25:38.993337Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
mysql_1              | 2021-01-29T07:25:39.133018Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
mysql_1              | 2021-01-29T07:25:39.133222Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
mysql_1              | 2021-01-29T07:25:39.140217Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
mysql_1              | 2021-01-29T07:25:39.158033Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.23'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.
...
harvester-api_1      | Running migrations...
mysql_1              | mbind: Operation not permitted
harvester-api_1      | INFO  [alembic.runtime.migration] Context impl MySQLImpl.
harvester-api_1      | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
harvester-api_1      | INFO  [alembic.runtime.migration] Running upgrade  -> 38a7f29de7c7, Initial db layout
...
mysql_1              | mbind: Operation not permitted
mysql_1              | mbind: Operation not permitted
harvester-worker_1   | [2021-01-29 07:35:58,772: ERROR/ForkPoolWorker-2] Task app.tasks.start_sequencer[4921b4c8-71f4-4ae7-94e0-584e6f962454] raised unexpected: ProgrammingError("(mysql.connector.errors.ProgrammingError) 1146 (42S02): Table 'polkascan.harvester_status' doesn't exist")
harvester-worker_1   | Traceback (most recent call last):
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
harvester-worker_1   |     self.dialect.do_execute(
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
harvester-worker_1   |     cursor.execute(statement, parameters)
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/mysql/connector/cursor.py", line 569, in execute
harvester-worker_1   |     self._handle_result(self._connection.cmd_query(stmt))
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/mysql/connector/connection.py", line 553, in cmd_query
harvester-worker_1   |     result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/mysql/connector/connection.py", line 442, in _handle_result
harvester-worker_1   |     raise errors.get_exception(packet)
harvester-worker_1   | mysql.connector.errors.ProgrammingError: 1146 (42S02): Table 'polkascan.harvester_status' doesn't exist
harvester-worker_1   |
harvester-worker_1   | The above exception was the direct cause of the following exception:
harvester-worker_1   |
harvester-worker_1   | Traceback (most recent call last):
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/celery/app/trace.py", line 385, in trace_task
harvester-worker_1   |     R = retval = fun(*args, **kwargs)
harvester-worker_1   |   File "/usr/src/app/app/tasks.py", line 70, in __call__
harvester-worker_1   |     return super().__call__(*args, **kwargs)
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/celery/app/trace.py", line 648, in __protected_call__
harvester-worker_1   |     return self.run(*args, **kwargs)
harvester-worker_1   |   File "/usr/src/app/app/tasks.py", line 154, in start_sequencer
harvester-worker_1   |     sequencer_task = Status.get_status(self.session, 'SEQUENCER_TASK_ID')
harvester-worker_1   |   File "/usr/src/app/app/models/harvester.py", line 34, in get_status
harvester-worker_1   |     model = session.query(cls).filter_by(key=key).first()
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3232, in first
harvester-worker_1   |     ret = list(self[0:1])
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3018, in __getitem__
harvester-worker_1   |     return list(res)
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3334, in __iter__
harvester-worker_1   |     return self._execute_and_instances(context)
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3359, in _execute_and_instances
harvester-worker_1   |     result = conn.execute(querycontext.statement, self._params)
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 988, in execute
harvester-worker_1   |     return meth(self, multiparams, params)
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
harvester-worker_1   |     return connection._execute_clauseelement(self, multiparams, params)
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1101, in _execute_clauseelement
harvester-worker_1   |     ret = self._execute_context(
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1252, in _execute_context
harvester-worker_1   |     self._handle_dbapi_exception(
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
harvester-worker_1   |     util.raise_from_cause(sqlalchemy_exception, exc_info)
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
harvester-worker_1   |     reraise(type(exception), exception, tb=exc_tb, cause=cause)
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
harvester-worker_1   |     raise value.with_traceback(tb)
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
harvester-worker_1   |     self.dialect.do_execute(
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
harvester-worker_1   |     cursor.execute(statement, parameters)
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/mysql/connector/cursor.py", line 569, in execute
harvester-worker_1   |     self._handle_result(self._connection.cmd_query(stmt))
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/mysql/connector/connection.py", line 553, in cmd_query
harvester-worker_1   |     result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
harvester-worker_1   |   File "/usr/local/lib/python3.8/site-packages/mysql/connector/connection.py", line 442, in _handle_result
harvester-worker_1   |     raise errors.get_exception(packet)
harvester-worker_1   | sqlalchemy.exc.ProgrammingError: (mysql.connector.errors.ProgrammingError) 1146 (42S02): Table 'polkascan.harvester_status' doesn't exist
harvester-worker_1   | [SQL: SELECT harvester_status.`key` AS harvester_status_key, harvester_status.value AS harvester_status_value, harvester_status.last_modified AS harvester_status_last_modified, harvester_status.notes AS harvester_status_notes
harvester-worker_1   | FROM harvester_status
harvester-worker_1   | WHERE harvester_status.`key` = %(key_1)s
harvester-worker_1   |  LIMIT %(param_1)s]
harvester-worker_1   | [parameters: {'key_1': 'SEQUENCER_TASK_ID', 'param_1': 1}]
harvester-worker_1   | (Background on this error at: http://sqlalche.me/e/f405)
...
harvester-worker_1   | mysql.connector.errors.ProgrammingError: 1146 (42S02): Table 'polkascan.runtime' doesn't exist

I guess it's not about caching_sha_2.

[phatblinkie]

i have the sha issue with centos 7, fresh install with the docker commands on the readme. tried all day to find a fix

[arjanz]

I have updated the Docker base image and requirements.txt to latest version (in main branch), does this solve the issue? Somehow I can't reproduce it.

[arjanz]

Quick note, don't forget to update the submodules as well with: git submodule update --init --recursive