search

polardb / polardbx-sql

PolarDB-X is a cloud native distributed SQL Database designed for high concurrency, massive storage, complex querying scenarios.
Apache License 2.0
1.54k stars 325 forks source link

pxd tryout fails #73

Closed yangyueren closed 2 years ago

yangyueren commented 2 years ago

Following https://www.polardbx.com/document , I run pxd tryout, but I got error below:

Digest: sha256:41d4f23930d5bce8d176944a784b6b40c79d97b7faf5423d2077a1ba526b7b8c
Status: Image is up to date for polardbx/polardbx-init:latest
Pull image: polardbx/xstore-tools:latest at 127.0.0.1

latest:Pulling from polardbx/xstore-tools 
Digest: sha256:5c0b19954c18fd2347eee46dffd82c4babf7e57b2e405d8b4ccb4d5a647c19a7
Status: Image is up to date for polardbx/xstore-tools:latest
Processing  [#########---------------------------]   25%    create gms node

Traceback (most recent call last):
  File "/workdir/polar-x/venv/lib64/python3.6/site-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/workdir/polar-x/venv/lib64/python3.6/site-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.30/containers/9175e58605d9cc143ee42c1e14f7f22b945f09b0fcc7f041e348da707abd9d4c/start

During handling of the above exception, another exception occurred:

...

raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.30/containers/9175e58605d9cc143ee42c1e14f7f22b945f09b0fcc7f041e348da707abd9d4c/start: Internal Server Error ("oci runtime error: container_linux.go:262: starting container process caused "process_linux.go:339: container init caused \"read init-p: connection reset by peer\""
")
vettalwu commented 2 years ago

@yangyueren Please provide your docker engine version and check docker daemon status.

yangyueren commented 2 years ago

docker daemon status.

$docker --version Docker version 17.06.0-ce, build 02c1d87

yangyueren commented 2 years ago

@yangyueren Please provide your docker engine version and check docker daemon status.

I succeed in my laptop with Docker version 20.10.16, build aa7e414

mazizang commented 1 year ago

i am using close-version docker (Docker version 20.10.12, build e91ed57) ummm..... i got same problem

mazizang commented 1 year ago

docker logs -f pxc-tryout-gms-Cand-15399 /tools/xstore/current/venv/lib64/python3.6/site-packages/pymysql/_auth.py:8: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6. from cryptography.hazmat.backends import default_backend 2023-08-07 02:33:39,871 - GalaxyEngine - INFO - begin wait for enable 2023-08-07 02:33:44,875 - GalaxyEngine - INFO - finish wait for enable INFO:root:Begin to initialize... 2023-08-07 02:33:44,885 - GalaxyEngine - INFO - () start command: /usr/bin/ln -sf /data/mysql-log/log /data/mysql/log 2023-08-07 02:33:44,888 - GalaxyEngine - INFO - succeeding to exec cmd ['/usr/bin/ln', '-sf', '/data/mysql-log/log', '/data/mysql/log'] 2023-08-07 02:33:44,889 - GalaxyEngine - INFO - (cwd=/data/mysql/data) execute command: /opt/galaxy_engine/bin/mysqld --defaults-file=/data/mysql/conf/my.cnf --loose-pod-name=pxc-tryout-gms-Cand-15399 --loose-cluster-info=172.17.0.2:23399@1 --initialize-insecure Traceback (most recent call last): File "/tools/xstore/current/entrypoint.py", line 119, in _start() File "/tools/xstore/current/venv/lib64/python3.6/site-packages/click/core.py", line 829, in call return self.main(args, kwargs) File "/tools/xstore/current/venv/lib64/python3.6/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/tools/xstore/current/venv/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "/tools/xstore/current/venv/lib64/python3.6/site-packages/click/core.py", line 610, in invoke return callback(args, **kwargs) File "/tools/xstore/current/entrypoint.py", line 99, in _start engine.initialize() File "/tools/xstore/v1.4.0-1681277870.050488/core/engine/galaxy/engine.py", line 151, in initialize self._initialize_data() File "/tools/xstore/v1.4.0-1681277870.050488/core/engine/galaxy/engine.py", line 141, in _initialize_data stderr=sys.stderr File "/tools/xstore/v1.4.0-1681277870.050488/core/engine/engine.py", line 324, in check_run_process return subprocess.check_call(cmd, cwd=cwd, stdout=stdout, stderr=stderr, env=self.context.subprocess_envs()) File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/opt/galaxy_engine/bin/mysqld', '--defaults-file=/data/mysql/conf/my.cnf', '--loose-pod-name=pxc-tryout-gms-Cand-15399', '--loose-cluster-info=172.17.0.2:23399@1', '--initialize-insecure']' returned non-zero exit status 1.

vettalwu commented 1 year ago

@mazizang The error message indicates the gms process initialize failed. Please check log file {user}/.pxd/log/polarx-log/{ip}_{port}/log/alert.log.

WangMinan commented 12 months ago

Same problem with @mazizang while creating the simple cluster with pxd tryout. Cannot create gms node. Docker version is Docker version 24.0.7, build afdd53b and the base system is centos7.9. The output of docker logs is as follows

INFO:root:Begin to initialize...
2023-10-31 15:39:03,435 - GalaxyEngine - INFO - (cwd=/data/mysql/data) execute command: /opt/galaxy_engine/bin/mysqld --defaults-file=/data/mysql/conf/my.cnf --loose-pod-name=pxc-tryout-gms-Cand-17863 --initialize-insecure
Traceback (most recent call last):
  File "/tools/xstore/current/entrypoint.py", line 96, in <module>
    _start()
  File "/tools/xstore/current/venv/lib64/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tools/xstore/current/venv/lib64/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tools/xstore/current/venv/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tools/xstore/current/venv/lib64/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tools/xstore/current/entrypoint.py", line 86, in _start
    engine.initialize()
  File "/tools/xstore/v1.0.0-1634524581.278688/core/engine/galaxy/engine.py", line 126, in initialize
    self._initialize_data()
  File "/tools/xstore/v1.0.0-1634524581.278688/core/engine/galaxy/engine.py", line 115, in _initialize_data
    cwd=self.path_data,
  File "/tools/xstore/v1.0.0-1634524581.278688/core/engine/engine.py", line 171, in check_run_process
    return subprocess.check_call(cmd, cwd=cwd, stdout=stdout, stderr=stderr, env=self.context.subprocess_envs())
  File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/opt/galaxy_engine/bin/mysqld', '--defaults-file=/data/mysql/conf/my.cnf', '--loose-pod-name=pxc-tryout-gms-Cand-17863', '--initialize-insecure']' returned non-zero exit status 1.

There is no log in {user}/.pxd/log/polarx-log/{ip}_{port}/log/

[root@ecs-polardbx .pxd]# tree -L 3
.
├── data
│   ├── cache
│   │   └── xcluster-tools
│   ├── podinfo
│   │   └── 127_0_0_1_17863
│   ├── polarx
│   │   └── 127_0_0_1_17863
│   ├── polarx-log
│   │   └── 127_0_0_1_17863
│   ├── shared
│   │   └── pxc-tryout-gms
│   └── template
│       └── my.cnf.template
└── polardbx.db

12 directories, 2 files
[root@ecs-polardbx .pxd]# cd data/polarx-log/
[root@ecs-polardbx polarx-log]# ls
127_0_0_1_17863
[root@ecs-polardbx polarx-log]# cd 127_0_0_1_17863/
[root@ecs-polardbx 127_0_0_1_17863]# ll
total 0
WangMinan commented 12 months ago

I finally found the following alert.log with mlocate

2023-10-31T08:12:58.367797Z 0 [Warning] [MY-011069] [Server] The syntax 'avoid_temporal_upgrade' is deprecated and will be removed in a future release.
2023-10-31T08:12:58.367851Z 0 [Warning] [MY-011068] [Server] The syntax 'expire-logs-days' is deprecated and will be removed in a future release. Please use binlog_expire_logs_seconds instead.
2023-10-31T08:12:58.367960Z 0 [Warning] [MY-011069] [Server] The syntax '--log-bin-use-v1-row-events' is deprecated and will be removed in a future release.
2023-10-31T08:12:58.368118Z 0 [Warning] [MY-011069] [Server] The syntax '--master-info-file' is deprecated and will be removed in a future release.
2023-10-31T08:12:58.368168Z 0 [Warning] [MY-011069] [Server] The syntax '--relay-log-info-file' is deprecated and will be removed in a future release.
2023-10-31T08:12:58.368184Z 0 [Warning] [MY-011069] [Server] The syntax 'show_old_temporals' is deprecated and will be removed in a future release.
2023-10-31T08:12:58.368236Z 0 [Warning] [MY-010084] [Server] options --log-slow-admin-statements, --log-queries-not-using-indexes and --log-slow-slave-statements have no effect if --slow-query-log is not set
2023-10-31T08:12:58.368240Z 0 [Warning] [MY-010086] [Server] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2023-10-31T08:12:58.368300Z 0 [System] [MY-013169] [Server] /opt/galaxy_engine/bin/mysqld (mysqld 8.0.18) initializing of server in progress as process 14
2023-10-31T08:12:58.369300Z 0 [Warning] [MY-013242] [Server] --character-set-server: 'utf8' is currently an alias for the character set UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please consider using UTF8MB4 in order to be unambiguous.
 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000
 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000
2023-10-31T08:13:31.242233Z 1 [Warning] [MY-011825] [InnoDB]   [GTID INFO] Reading from undo log :
2023-10-31T08:13:32.107488Z 0 [Warning] [MY-000000] [Server] Plugin polarx_rpc reported: 'PolarX RPC disabled by database initialization.'
2023-10-31T08:13:32.107690Z 0 [Warning] [MY-011237] [Server] Plugin mysqlx reported: 'Startup failed with error "Galaxy X Protocol disabled when mysql initialize."'
2023-10-31T08:13:32.457952Z 0 [Warning] [MY-010015] [Repl] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
2023-10-31T08:13:32.919404Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_extra_max_connections=65535'.
2023-10-31T08:13:32.919736Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_galaxy_max_connections=4096'.
2023-10-31T08:13:32.920074Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_implicit_primary_key=1'.
2023-10-31T08:13:32.920383Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_innodb_doublewrite_pages=64'.
2023-10-31T08:13:32.920692Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_innodb_log_optimize_ddl=OFF'.
2023-10-31T08:13:32.921029Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_innodb_multi_blocks_enabled=ON'.
2023-10-31T08:13:32.921347Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_innodb_numa_interleave=ON'.
2023-10-31T08:13:32.921663Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_innodb_primary_flush_max_lsn_lag=122016116'.
2023-10-31T08:13:32.922001Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_innodb_replica_log_parse_buf_size=134217728'.
2023-10-31T08:13:32.922234Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_log_sql_fifo=/data/mysql/run/mysql.fifo'.
2023-10-31T08:13:32.922435Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_persist_binlog_to_redo=OFF'.
2023-10-31T08:13:32.922628Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose_persist_binlog_to_redo_size_limit=1048576'.
2023-10-31T08:13:32.922927Z 0 [Warning] [MY-000067] [Server] unknown variable 'loose-pod-name=pxc-tryout-gms-Cand-15336'.
2023-10-31T08:13:32.930111Z 0 [ERROR] [MY-000000] [Server] PolarDB-X Engine cluster_info must be set when the server is running with --initialize(-insecure)
2023-10-31T08:13:32.930359Z 0 [ERROR] [MY-013236] [Server] The designated data directory /data/mysql/data/ is unusable. You can remove all files that the server added to it.
2023-10-31T08:13:32.930698Z 0 [ERROR] [MY-010119] [Server] Aborting
2023-10-31T08:13:34.038346Z 0 [System] [MY-010910] [Server] /opt/galaxy_engine/bin/mysqld: Shutdown complete (mysqld 8.0.18)  Source distribution

Sorry but what's wrong?