Closed jmontleon closed 3 years ago
jmontleon
commented
3 years ago Does removing the storageClassName break something that was working previously? If we need to start doing multiple replacements (OCP 3 vs OCP 4, storageClassName, etc.) perhaps we should be doing this with ansible templates rather than a build script that sews a bunch of static files together...
shawn-hurley
commented
3 years ago As we learn more about each cloud and it's storage class offerings and dealing with multi-az, I wonder if it maybe makes sense to transition to something more suited for templating. I think maybe that should be a follow up to this PR though? WDYT?
ashoks27
commented
3 years ago Hello @shawn-hurley @jmontleon Thanks for the update. I have tried with latest version of robot-shop manifest files to deploy it in the ROKS cluster and I am still getting the errors. Have updated the api version as v1 for deployments and updated the scc as anyuid. Please check the logs below .
[root@oc4377604745 robot-shop]# oc get pods NAME READY STATUS RESTARTS AGE cart-699567cb75-x962p 1/1 Running 0 4m1s catalogue-7476fdc8f4-fc6mj 1/1 Running 0 4m dispatch-85ff479cdd-7q5nt 1/1 Running 0 4m mongodb-59b795bd6b-zkx7c 1/1 Running 0 3m59s mysql-6dd8fd6c8d-j47bb 1/2 Error 4 3m58s payment-695779d6bb-l9lfz 1/1 Running 0 3m57s rabbitmq-7b44f9c449-6m6qt 1/1 Running 0 3m56s ratings-c45895c99-v7kqq 1/1 Running 0 3m56s redis-78dd47d8c-qqr24 1/1 Running 0 3m55s shipping-6fbbc47b67-f47hh 1/1 Running 0 3m54s user-56987c97fd-9k6c6 1/1 Running 0 3m53s web-ffbbb68c7-df4wv 1/1 Running 0 3m53s [root@oc4377604745 robot-shop]# [root@oc4377604745 robot-shop]# [root@oc4377604745 robot-shop]# [root@oc4377604745 robot-shop]# oc logs -f mysql-6dd8fd6c8d-j47bb Error from server (BadRequest): a container name must be specified for pod mysql-6dd8fd6c8d-j47bb, choose one of: [dataloader mysql] or one of the init containers: [post-hook] [root@oc4377604745 robot-shop]# [root@oc4377604745 robot-shop]# [root@oc4377604745 robot-shop]# oc logs -f mysql-6dd8fd6c8d-j47bb -c mysql => sourcing 20-validate-variables.sh ... => sourcing 25-validate-replication-variables.sh ... => sourcing 30-base-config.sh ... ---> 08:35:20 Processing basic MySQL configuration files ... => sourcing 60-replication-config.sh ... => sourcing 70-s2i-config.sh ... ---> 08:35:20 Processing additional arbitrary MySQL configuration provided by s2i ... => sourcing 10-mysql57.cnf ... => sourcing 40-paas.cnf ... => sourcing 50-my-tuning.cnf ... ---> 08:35:20 Initializing database ... ---> 08:35:20 Running /opt/rh/rh-mysql57/root/usr/libexec/mysqld --initialize --datadir=/var/lib/mysql/data --ignore-db-dir=lost+found ---> 08:35:20 Starting MySQL server with disabled networking ... ---> 08:35:20 Waiting for MySQL to start ... 2020-11-17T08:35:21.646880Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2020-11-17T08:35:21.647971Z 0 [Warning] Can't create test file /var/lib/mysql/data/mysql-6dd8fd6c8d-j47bb.lower-test 2020-11-17T08:35:21.648134Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld (mysqld 5.7.24) starting as process 40 ... 2020-11-17T08:35:21.653283Z 0 [Warning] Can't create test file /var/lib/mysql/data/mysql-6dd8fd6c8d-j47bb.lower-test 2020-11-17T08:35:21.653714Z 0 [Warning] Can't create test file /var/lib/mysql/data/mysql-6dd8fd6c8d-j47bb.lower-test 2020-11-17T08:35:21.749337Z 0 [Note] InnoDB: PUNCH HOLE support available 2020-11-17T08:35:21.749518Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2020-11-17T08:35:21.749584Z 0 [Note] InnoDB: Uses event mutexes 2020-11-17T08:35:21.749649Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier 2020-11-17T08:35:21.749711Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.7 2020-11-17T08:35:21.749778Z 0 [Note] InnoDB: Using Linux native AIO 2020-11-17T08:35:21.750335Z 0 [Note] InnoDB: Number of pools: 1 2020-11-17T08:35:21.750601Z 0 [Note] InnoDB: Using CPU crc32 instructions 2020-11-17T08:35:21.753792Z 0 [Note] InnoDB: Initializing buffer pool, total size = 512M, instances = 1, chunk size = 128M 2020-11-17T08:35:22.046731Z 0 [Note] InnoDB: Completed initialization of buffer pool ---> 08:35:22 Waiting for MySQL to start ... 2020-11-17T08:35:22.235610Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2020-11-17T08:35:22.246073Z 0 [Note] InnoDB: The first innodb_system data file 'ibdata1' did not exist. A new tablespace will be created! 2020-11-17T08:35:22.248107Z 0 [ERROR] InnoDB: Operating system error number 13 in a file operation. 2020-11-17T08:35:22.248135Z 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory. 2020-11-17T08:35:22.248143Z 0 [ERROR] InnoDB: Operating system error number 13 in a file operation. 2020-11-17T08:35:22.248149Z 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory. 2020-11-17T08:35:22.248159Z 0 [ERROR] InnoDB: Cannot open datafile './ibdata1' 2020-11-17T08:35:22.248172Z 0 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contain your precious data! 2020-11-17T08:35:22.248183Z 0 [ERROR] InnoDB: InnoDB Database creation was aborted with error Cannot open a file. You may need to delete the ibdata1 file before trying to start up again. 2020-11-17T08:35:22.848858Z 0 [ERROR] Plugin 'InnoDB' init function returned error. 2020-11-17T08:35:22.849035Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2020-11-17T08:35:22.849162Z 0 [ERROR] Failed to initialize builtin plugins. 2020-11-17T08:35:22.849224Z 0 [ERROR] Aborting
2020-11-17T08:35:22.849303Z 0 [Note] Binlog end 2020-11-17T08:35:22.849486Z 0 [Note] Shutting down plugin 'MyISAM' 2020-11-17T08:35:22.849569Z 0 [Note] Shutting down plugin 'CSV' 2020-11-17T08:35:22.855503Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld: Shutdown complete
jmontleon
commented
3 years ago @ashoks27 you shouldn't have to modify anything manually. Just run builld.sh 4 and deploy.sh.
[jmontleo@jmontleo robot-shop]$ ./build.sh 4
[jmontleo@jmontleo robot-shop]$ ./deploy.sh
project.project.openshift.io/robot-shop created
deployment.apps/cart created
service/cart created
deployment.apps/catalogue created
service/catalogue created
deployment.apps/dispatch created
service/dispatch created
deployment.apps/mongodb created
persistentvolumeclaim/mongodb-volume-claim created
service/mongodb created
deployment.apps/mysql created
persistentvolumeclaim/mysql-data-volume-claim created
service/mysql created
deployment.apps/payment created
service/payment created
deployment.apps/rabbitmq created
service/rabbitmq created
deployment.apps/ratings created
service/ratings created
deployment.apps/redis created
persistentvolumeclaim/redis-volume-claim created
service/redis created
deployment.apps/shipping created
service/shipping created
deployment.apps/user created
service/user created
deployment.apps/web created
route.route.openshift.io/front-end created
service/web created[jmontleo@jmontleo robot-shop]$ oc get po -n robot-shop
NAME READY STATUS RESTARTS AGE
cart-56d64d845-m6mw8 1/1 Running 0 2m30s
catalogue-7b84494bf6-b58m9 1/1 Running 0 2m30s
dispatch-86bd8f4886-qg47l 1/1 Running 0 2m30s
mongodb-ffd467b6d-q8xtr 1/1 Running 0 2m30s
mysql-64f679d7b6-zmmpk 2/2 Running 0 2m30s
payment-74fffc4db4-cd26p 1/1 Running 0 2m29s
rabbitmq-5bb66bb6c9-lr2lm 1/1 Running 0 2m29s
ratings-65964c5f8f-pkmm5 1/1 Running 0 2m29s
redis-5694467c97-cpqbm 1/1 Running 0 2m29s
shipping-b798968df-dbrk6 1/1 Running 0 2m29s
user-84496cc588-pb9sq 1/1 Running 0 2m29s
web-79cb976f9-86rjx 1/1 Running 0 2m28s[jmontleo@jmontleo robot-shop]$ oc logs -n robot-shop -c mysql mysql-64f679d7b6-zmmpk
=> sourcing 20-validate-variables.sh ...
=> sourcing 25-validate-replication-variables.sh ...
=> sourcing 30-base-config.sh ...
---> 14:24:02 Processing basic MySQL configuration files ...
=> sourcing 60-replication-config.sh ...
=> sourcing 70-s2i-config.sh ...
---> 14:24:02 Processing additional arbitrary MySQL configuration provided by s2i ...
=> sourcing 10-mysql57.cnf ...
=> sourcing 40-paas.cnf ...
=> sourcing 50-my-tuning.cnf ...
---> 14:24:02 Initializing database ...
---> 14:24:02 Running /opt/rh/rh-mysql57/root/usr/libexec/mysqld --initialize --datadir=/var/lib/mysql/data --ignore-db-dir=lost+found
---> 14:24:35 Starting MySQL server with disabled networking ...
---> 14:24:35 Waiting for MySQL to start ...
2020-11-19T14:24:35.611656Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2020-11-19T14:24:35.692190Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld (mysqld 5.7.24) starting as process 66 ...
2020-11-19T14:24:35.706612Z 0 [Note] InnoDB: PUNCH HOLE support available
2020-11-19T14:24:35.706764Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-11-19T14:24:35.706804Z 0 [Note] InnoDB: Uses event mutexes
2020-11-19T14:24:35.706843Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2020-11-19T14:24:35.706878Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
2020-11-19T14:24:35.706922Z 0 [Note] InnoDB: Using Linux native AIO
2020-11-19T14:24:35.707443Z 0 [Note] InnoDB: Number of pools: 1
2020-11-19T14:24:35.707681Z 0 [Note] InnoDB: Using CPU crc32 instructions
2020-11-19T14:24:35.710076Z 0 [Note] InnoDB: Initializing buffer pool, total size = 512M, instances = 1, chunk size = 128M
2020-11-19T14:24:35.903453Z 0 [Note] InnoDB: Completed initialization of buffer pool
2020-11-19T14:24:35.992890Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-11-19T14:24:36.005448Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2020-11-19T14:24:36.095001Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-11-19T14:24:36.095374Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
---> 14:24:36 Waiting for MySQL to start ...
2020-11-19T14:24:36.565191Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-11-19T14:24:36.566153Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2020-11-19T14:24:36.566245Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2020-11-19T14:24:36.566778Z 0 [Note] InnoDB: Waiting for purge to start
2020-11-19T14:24:36.617036Z 0 [Note] InnoDB: 5.7.24 started; log sequence number 2592375
2020-11-19T14:24:36.617658Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/data/ib_buffer_pool
2020-11-19T14:24:36.617785Z 0 [Note] Plugin 'FEDERATED' is disabled.
2020-11-19T14:24:36.619591Z 0 [Note] InnoDB: Buffer pool(s) load completed at 201119 14:24:36
2020-11-19T14:24:36.622418Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
2020-11-19T14:24:36.622444Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
2020-11-19T14:24:36.623122Z 0 [Warning] CA certificate ca.pem is self signed.
2020-11-19T14:24:36.623177Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
2020-11-19T14:24:36.626082Z 0 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:36.626147Z 0 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:36.626163Z 0 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:36.626192Z 0 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:36.626203Z 0 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:36.626217Z 0 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:36.627707Z 0 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:36.627729Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:36.633819Z 0 [Note] Event Scheduler: Loaded 0 events
2020-11-19T14:24:36.634035Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld: ready for connections.
Version: '5.7.24' socket: '/tmp/mysql.sock' port: 0 MySQL Community Server (GPL)
2020-11-19T14:24:37.232344Z 2 [Note] Your password has expired. To log in you must change it using a client that supports expired passwords.
---> 14:24:37 MySQL started successfully
---> 14:24:37 Setting password for MySQL root user ...
mysql: [Warning] Using a password on the command line interface can be insecure.
Checking if update is needed.
Checking server version.
Running queries to upgrade MySQL server.
2020-11-19T14:24:39.367265Z 5 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:39.367397Z 5 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:39.367446Z 5 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:39.367512Z 5 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:39.367553Z 5 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:39.367624Z 5 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:39.367723Z 5 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:39.367873Z 5 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.547745Z 5 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.547899Z 5 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.547944Z 5 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.547999Z 5 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.548032Z 5 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.548077Z 5 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.548185Z 5 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.548223Z 5 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
Checking system database.
mysql.columns_priv OK
mysql.db OK
mysql.engine_cost OK
mysql.event OK
mysql.func OK
mysql.general_log OK
mysql.gtid_executed OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_relation OK
mysql.help_topic OK
mysql.innodb_index_stats OK
mysql.innodb_table_stats OK
mysql.ndb_binlog_index OK
mysql.plugin OK
mysql.proc OK
mysql.procs_priv OK
mysql.proxies_priv OK
mysql.server_cost OK
mysql.servers OK
mysql.slave_master_info OK
mysql.slave_relay_log_info OK
mysql.slave_worker_info OK
mysql.slow_log OK
mysql.tables_priv OK
mysql.time_zone OK
mysql.time_zone_leap_second OK
mysql.time_zone_name OK
mysql.time_zone_transition OK
mysql.time_zone_transition_type OK
mysql.user OK
The sys schema is already up to date (version 1.5.1).
Checking databases.
sys.sys_config OK
Upgrade process completed successfully.
Checking if update is needed.
---> 14:24:43 Creating user specified by MYSQL_USER (shipping) ...
---> 14:24:43 Creating database cities ...
---> 14:24:43 Granting privileges to user shipping for cities ...
2020-11-19T14:24:43.807179Z 8 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.807214Z 8 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.807222Z 8 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.807244Z 8 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.807249Z 8 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.807261Z 8 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.807411Z 8 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:43.807424Z 8 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
---> 14:24:43 Initialization finished
=> sourcing 40-datadir-action.sh ...
---> 14:24:44 Running datadir action: upgrade-warn
---> 14:24:44 MySQL server version check passed, both server and data directory are version 5.7.
=> sourcing 50-passwd-change.sh ...
---> 14:24:44 Setting passwords ...
2020-11-19T14:24:44.201792Z 11 [Warning] Following users were specified in CREATE USER IF NOT EXISTS but they already exist. Corresponding entry in binary log used default authentication plugin 'mysql_native_password' to rewrite authentication information(if any) for them: 'root'@'%'
---> 14:24:44 Shutting down MySQL ...
2020-11-19T14:24:44.292215Z 12 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:44.292254Z 12 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:44.292262Z 12 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:44.292283Z 12 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:44.292288Z 12 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:44.292300Z 12 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:44.292496Z 12 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:44.292510Z 12 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:44.296270Z 0 [Note] Giving 0 client threads a chance to die gracefully
2020-11-19T14:24:44.296326Z 0 [Note] Shutting down slave threads
2020-11-19T14:24:44.296363Z 0 [Note] Forcefully disconnecting 0 remaining clients
2020-11-19T14:24:44.296398Z 0 [Note] Event Scheduler: Purging the queue. 0 events
2020-11-19T14:24:44.296478Z 0 [Note] Binlog end
2020-11-19T14:24:44.298823Z 0 [Note] Shutting down plugin 'ngram'
2020-11-19T14:24:44.298846Z 0 [Note] Shutting down plugin 'BLACKHOLE'
2020-11-19T14:24:44.298853Z 0 [Note] Shutting down plugin 'ARCHIVE'
2020-11-19T14:24:44.298856Z 0 [Note] Shutting down plugin 'partition'
2020-11-19T14:24:44.298859Z 0 [Note] Shutting down plugin 'INNODB_SYS_VIRTUAL'
2020-11-19T14:24:44.298863Z 0 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2020-11-19T14:24:44.298866Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES'
2020-11-19T14:24:44.298869Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
2020-11-19T14:24:44.298872Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
2020-11-19T14:24:44.298875Z 0 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
2020-11-19T14:24:44.298877Z 0 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
2020-11-19T14:24:44.298880Z 0 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
2020-11-19T14:24:44.298883Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
2020-11-19T14:24:44.298886Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
2020-11-19T14:24:44.298888Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
2020-11-19T14:24:44.298892Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
2020-11-19T14:24:44.298895Z 0 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
2020-11-19T14:24:44.298897Z 0 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
2020-11-19T14:24:44.298899Z 0 [Note] Shutting down plugin 'INNODB_FT_DELETED'
2020-11-19T14:24:44.298903Z 0 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
2020-11-19T14:24:44.298906Z 0 [Note] Shutting down plugin 'INNODB_METRICS'
2020-11-19T14:24:44.298908Z 0 [Note] Shutting down plugin 'INNODB_TEMP_TABLE_INFO'
2020-11-19T14:24:44.298911Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
2020-11-19T14:24:44.298914Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
2020-11-19T14:24:44.298917Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
2020-11-19T14:24:44.298919Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET'
2020-11-19T14:24:44.298922Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
2020-11-19T14:24:44.298925Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
2020-11-19T14:24:44.298928Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM'
2020-11-19T14:24:44.298931Z 0 [Note] Shutting down plugin 'INNODB_CMP_RESET'
2020-11-19T14:24:44.298933Z 0 [Note] Shutting down plugin 'INNODB_CMP'
2020-11-19T14:24:44.298936Z 0 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
2020-11-19T14:24:44.298939Z 0 [Note] Shutting down plugin 'INNODB_LOCKS'
2020-11-19T14:24:44.298942Z 0 [Note] Shutting down plugin 'INNODB_TRX'
2020-11-19T14:24:44.298944Z 0 [Note] Shutting down plugin 'InnoDB'
2020-11-19T14:24:44.299055Z 0 [Note] InnoDB: FTS optimize thread exiting.
2020-11-19T14:24:44.299286Z 0 [Note] InnoDB: Starting shutdown...
2020-11-19T14:24:44.399517Z 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/data/ib_buffer_pool
2020-11-19T14:24:44.400135Z 0 [Note] InnoDB: Buffer pool(s) dump completed at 201119 14:24:44
2020-11-19T14:24:45.836823Z 0 [Note] InnoDB: Shutdown completed; log sequence number 2691184
2020-11-19T14:24:45.838979Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2020-11-19T14:24:45.839007Z 0 [Note] Shutting down plugin 'MEMORY'
2020-11-19T14:24:45.839014Z 0 [Note] Shutting down plugin 'MRG_MYISAM'
2020-11-19T14:24:45.839024Z 0 [Note] Shutting down plugin 'MyISAM'
2020-11-19T14:24:45.839034Z 0 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2020-11-19T14:24:45.839073Z 0 [Note] Shutting down plugin 'CSV'
2020-11-19T14:24:45.839085Z 0 [Note] Shutting down plugin 'sha256_password'
2020-11-19T14:24:45.839088Z 0 [Note] Shutting down plugin 'mysql_native_password'
2020-11-19T14:24:45.839238Z 0 [Note] Shutting down plugin 'binlog'
2020-11-19T14:24:45.890168Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld: Shutdown complete
---> 14:24:46 Cleaning up environment variables MYSQL_USER, MYSQL_PASSWORD, MYSQL_DATABASE and MYSQL_ROOT_PASSWORD ...
---> 14:24:46 Running final exec -- Only MySQL server logs after this point
2020-11-19T14:24:46.898936Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2020-11-19T14:24:46.901336Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld (mysqld 5.7.24) starting as process 1 ...
2020-11-19T14:24:46.993826Z 0 [Note] InnoDB: PUNCH HOLE support available
2020-11-19T14:24:46.994000Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-11-19T14:24:46.994036Z 0 [Note] InnoDB: Uses event mutexes
2020-11-19T14:24:46.994070Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2020-11-19T14:24:46.994125Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
2020-11-19T14:24:46.994165Z 0 [Note] InnoDB: Using Linux native AIO
2020-11-19T14:24:46.994529Z 0 [Note] InnoDB: Number of pools: 1
2020-11-19T14:24:46.994699Z 0 [Note] InnoDB: Using CPU crc32 instructions
2020-11-19T14:24:46.996773Z 0 [Note] InnoDB: Initializing buffer pool, total size = 512M, instances = 1, chunk size = 128M
2020-11-19T14:24:47.109486Z 0 [Note] InnoDB: Completed initialization of buffer pool
2020-11-19T14:24:47.199901Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-11-19T14:24:47.212607Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2020-11-19T14:24:47.302810Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-11-19T14:24:47.303088Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-11-19T14:24:47.847376Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-11-19T14:24:47.848448Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2020-11-19T14:24:47.848518Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2020-11-19T14:24:47.849152Z 0 [Note] InnoDB: Waiting for purge to start
2020-11-19T14:24:47.899436Z 0 [Note] InnoDB: 5.7.24 started; log sequence number 2691184
2020-11-19T14:24:47.899848Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/data/ib_buffer_pool
2020-11-19T14:24:47.899990Z 0 [Note] Plugin 'FEDERATED' is disabled.
2020-11-19T14:24:47.901821Z 0 [Note] InnoDB: Buffer pool(s) load completed at 201119 14:24:47
2020-11-19T14:24:47.903956Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
2020-11-19T14:24:47.904158Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
2020-11-19T14:24:47.904851Z 0 [Warning] CA certificate ca.pem is self signed.
2020-11-19T14:24:47.904931Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
2020-11-19T14:24:47.905480Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
2020-11-19T14:24:47.905589Z 0 [Note] IPv6 is available.
2020-11-19T14:24:47.905627Z 0 [Note] - '::' resolves to '::';
2020-11-19T14:24:47.905690Z 0 [Note] Server socket created on IP: '::'.
2020-11-19T14:24:47.908972Z 0 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:47.909050Z 0 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:47.909089Z 0 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:47.909170Z 0 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:47.909202Z 0 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:47.909245Z 0 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:47.910179Z 0 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:47.910241Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-11-19T14:24:47.915827Z 0 [Note] Event Scheduler: Loaded 0 events
2020-11-19T14:24:47.916087Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld: ready for connections.
Version: '5.7.24' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Server (GPL)Hi @jmontleon , Thanks for the update . I have run build and deploy as you mentioned and still got the error. Herewith I have attached the logs for your quick reference . Please check . Thanks
[root@oc8660015353 robot-shop]# ./build.sh 4 [root@oc8660015353 robot-shop]# ./deploy.sh project.project.openshift.io/robot-shop created deployment.apps/cart created service/cart created deployment.apps/catalogue created service/catalogue created deployment.apps/dispatch created service/dispatch created deployment.apps/mongodb created persistentvolumeclaim/mongodb-volume-claim created service/mongodb created deployment.apps/mysql created persistentvolumeclaim/mysql-data-volume-claim created service/mysql created deployment.apps/payment created service/payment created deployment.apps/rabbitmq created service/rabbitmq created deployment.apps/ratings created service/ratings created deployment.apps/redis created persistentvolumeclaim/redis-volume-claim created service/redis created deployment.apps/shipping created service/shipping created deployment.apps/user created service/user created deployment.apps/web created route.route.openshift.io/front-end created service/web created
[root@oc8660015353 robot-shop]# oc get pods NAME READY STATUS RESTARTS AGE cart-699567cb75-lmjv7 1/1 Running 0 2m9s catalogue-7476fdc8f4-jzrbg 1/1 Running 0 2m8s dispatch-85ff479cdd-2nnvp 1/1 Running 0 2m7s mongodb-59b795bd6b-kq52s 1/1 Running 0 2m7s mysql-6dd8fd6c8d-cv7zs 1/2 Error 1 2m6s payment-695779d6bb-qx7f2 1/1 Running 0 2m5s rabbitmq-7b44f9c449-nj5ds 1/1 Running 0 2m4s ratings-c45895c99-22t64 1/1 Running 0 2m4s redis-78dd47d8c-p845m 1/1 Running 0 2m3s shipping-6fbbc47b67-xhqmq 1/1 Running 0 2m2s user-56987c97fd-rmt95 1/1 Running 0 2m2s web-ffbbb68c7-kfjjf 1/1 Running 0 2m1s [root@oc8660015353 robot-shop]# oc logs mysql-6dd8fd6c8d-cv7zs Error from server (BadRequest): a container name must be specified for pod mysql-6dd8fd6c8d-cv7zs, choose one of: [dataloader mysql] or one of the init containers: [post-hook] [root@oc8660015353 robot-shop]# oc logs mysql-6dd8fd6c8d-cv7zs -c post-hook [root@oc8660015353 robot-shop]# oc logs mysql-6dd8fd6c8d-cv7zs -c dataloader [root@oc8660015353 robot-shop]# oc logs mysql-6dd8fd6c8d-cv7zs -c mysql => sourcing 20-validate-variables.sh ... => sourcing 25-validate-replication-variables.sh ... => sourcing 30-base-config.sh ... ---> 12:01:04 Processing basic MySQL configuration files ... => sourcing 60-replication-config.sh ... => sourcing 70-s2i-config.sh ... ---> 12:01:04 Processing additional arbitrary MySQL configuration provided by s2i ... => sourcing 10-mysql57.cnf ... => sourcing 40-paas.cnf ... => sourcing 50-my-tuning.cnf ... ---> 12:01:04 Initializing database ... ---> 12:01:04 Running /opt/rh/rh-mysql57/root/usr/libexec/mysqld --initialize --datadir=/var/lib/mysql/data --ignore-db-dir=lost+found ---> 12:01:04 Starting MySQL server with disabled networking ... ---> 12:01:05 Waiting for MySQL to start ... 2020-11-20T12:01:05.747975Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2020-11-20T12:01:05.748989Z 0 [Warning] Can't create test file /var/lib/mysql/data/mysql-6dd8fd6c8d-cv7zs.lower-test 2020-11-20T12:01:05.749168Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld (mysqld 5.7.24) starting as process 40 ... 2020-11-20T12:01:05.754302Z 0 [Warning] Can't create test file /var/lib/mysql/data/mysql-6dd8fd6c8d-cv7zs.lower-test 2020-11-20T12:01:05.754695Z 0 [Warning] Can't create test file /var/lib/mysql/data/mysql-6dd8fd6c8d-cv7zs.lower-test 2020-11-20T12:01:05.850255Z 0 [Note] InnoDB: PUNCH HOLE support available 2020-11-20T12:01:05.850454Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2020-11-20T12:01:05.850519Z 0 [Note] InnoDB: Uses event mutexes 2020-11-20T12:01:05.850582Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier 2020-11-20T12:01:05.850644Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.7 2020-11-20T12:01:05.850711Z 0 [Note] InnoDB: Using Linux native AIO 2020-11-20T12:01:05.851261Z 0 [Note] InnoDB: Number of pools: 1 2020-11-20T12:01:05.851516Z 0 [Note] InnoDB: Using CPU crc32 instructions 2020-11-20T12:01:05.854672Z 0 [Note] InnoDB: Initializing buffer pool, total size = 512M, instances = 1, chunk size = 128M 2020-11-20T12:01:06.145135Z 0 [Note] InnoDB: Completed initialization of buffer pool ---> 12:01:06 Waiting for MySQL to start ... 2020-11-20T12:01:06.244582Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2020-11-20T12:01:06.256949Z 0 [Note] InnoDB: The first innodb_system data file 'ibdata1' did not exist. A new tablespace will be created! 2020-11-20T12:01:06.258752Z 0 [ERROR] InnoDB: Operating system error number 13 in a file operation. 2020-11-20T12:01:06.258782Z 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory. 2020-11-20T12:01:06.258790Z 0 [ERROR] InnoDB: Operating system error number 13 in a file operation. 2020-11-20T12:01:06.258796Z 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory. 2020-11-20T12:01:06.258807Z 0 [ERROR] InnoDB: Cannot open datafile './ibdata1' 2020-11-20T12:01:06.258819Z 0 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contain your precious data! 2020-11-20T12:01:06.258830Z 0 [ERROR] InnoDB: InnoDB Database creation was aborted with error Cannot open a file. You may need to delete the ibdata1 file before trying to start up again. 2020-11-20T12:01:06.859522Z 0 [ERROR] Plugin 'InnoDB' init function returned error. 2020-11-20T12:01:06.859565Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2020-11-20T12:01:06.859589Z 0 [ERROR] Failed to initialize builtin plugins. 2020-11-20T12:01:06.859596Z 0 [ERROR] Aborting
2020-11-20T12:01:06.859617Z 0 [Note] Binlog end 2020-11-20T12:01:06.859740Z 0 [Note] Shutting down plugin 'MyISAM' 2020-11-20T12:01:06.859766Z 0 [Note] Shutting down plugin 'CSV' 2020-11-20T12:01:06.864870Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld: Shutdown complete robot-shop-roks_detailed_logs.txt
ashoks27
commented
3 years ago Hello @jmontleon . Did you get a chance to check on the above issue .
Stop using execNewPod so we can use RWO storage. ROKS block storage does not support RWX and ROKS file storage requires running containers as root or in a cluster without multiple availability zones.
I also added a down and dirty flag to the build script so it will create a manifest.yaml usable on current OCP 4 releases.