Open vikramsisodia27 opened 3 years ago
I had the same problem.
macOS 10.15.5
cdc-service log
It's likely that the CDC is being killed because the Docker VM does not have enough memory.
@cer I am also facing error what @diguage facing. Increased processor to 5, RAM 5GB, SWAP 2GB in Mac but same result ./gradle :composeup removed all of my existing DB Schemas
@cer I am also facing error what @diguage facing. Increased processor to 5, RAM 5GB, SWAP 2GB in Mac but same result ./gradle :composeup removed all of my existing DB Schemas
When you "same error" - are you saying that the CDC container log says 'Killed'?
What's the output of docker ps -a
?
I reconfigured by Docker for Mac VM to be "processor to 5, RAM 5GB, SWAP 2GB" and the application started - I think it took at least 60 seconds, with services starting -> unhealthy -> healthy
Container is not killed -
ftgo-application_cdc-service_1 logs
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.GeneratedConstructorAccessor53.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:988)
at com.mysql.jdbc.MysqlIO.
Thanks for the logs.
The CDC cannot connect to the database.
This is the problem: 502768da80f6 ftgo-application_mysql "docker-entrypoint.s…" 2 minutes ago Exited (1) About a minute ago ftgo-application_mysql_1
What's the output of docker logs ftgo-application_mysql_1
?
Initializing database
2020-07-04T19:25:32.080069Z 0 [Warning] InnoDB: New log files created, LSN=45790
2020-07-04T19:25:32.174369Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2020-07-04T19:25:32.290518Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 2098312e-be2c-11ea-a820-0242ac130006.
2020-07-04T19:25:32.303226Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
2020-07-04T19:25:32.309891Z 1 [Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2020-07-04T19:25:34.208124Z 1 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:25:34.208866Z 1 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:25:34.208912Z 1 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:25:34.208946Z 1 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:25:34.209036Z 1 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
Database initialized
MySQL init process in progress...
2020-07-04T19:25:37.015630Z 0 [Note] mysqld (mysqld 5.7.13-log) starting as process 50 ...
2020-07-04T19:25:37.033382Z 0 [Note] InnoDB: PUNCH HOLE support available
2020-07-04T19:25:37.033440Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-07-04T19:25:37.033448Z 0 [Note] InnoDB: Uses event mutexes
2020-07-04T19:25:37.033457Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2020-07-04T19:25:37.033466Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.8
2020-07-04T19:25:37.033474Z 0 [Note] InnoDB: Using Linux native AIO
2020-07-04T19:25:37.038471Z 0 [Note] InnoDB: Number of pools: 1
2020-07-04T19:25:37.038745Z 0 [Note] InnoDB: Using CPU crc32 instructions
2020-07-04T19:25:37.040976Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2020-07-04T19:25:37.090437Z 0 [Note] InnoDB: Completed initialization of buffer pool
2020-07-04T19:25:37.095437Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-07-04T19:25:37.121972Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2020-07-04T19:25:37.191981Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-07-04T19:25:37.192126Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-07-04T19:25:37.316449Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-07-04T19:25:37.318566Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2020-07-04T19:25:37.318609Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2020-07-04T19:25:37.319387Z 0 [Note] InnoDB: Waiting for purge to start
2020-07-04T19:25:37.369860Z 0 [Note] InnoDB: 5.7.13 started; log sequence number 2525487
2020-07-04T19:25:37.373680Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2020-07-04T19:25:37.373680Z 0 [Note] Plugin 'FEDERATED' is disabled.
2020-07-04T19:25:37.384538Z 0 [Note] InnoDB: Buffer pool(s) load completed at 200704 19:25:37
2020-07-04T19:25:37.392080Z 0 [Warning] Failed to set up SSL because of the following SSL library error: SSL context is not usable without certificate and private key
2020-07-04T19:25:37.396169Z 0 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:25:37.396223Z 0 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:25:37.396250Z 0 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:25:37.396340Z 0 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:25:37.398158Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:25:37.553511Z 0 [Note] Event Scheduler: Loaded 0 events
2020-07-04T19:25:37.558100Z 0 [Note] mysqld: ready for connections.
Version: '5.7.13-log' socket: '/var/run/mysqld/mysqld.sock' port: 0 MySQL Community Server (GPL)
Warning: Unable to load '/usr/share/zoneinfo/Factory' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/posix/Factory' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/right/Factory' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
2020-07-04T19:25:53.730783Z 4 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:25:53.730855Z 4 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:25:53.730962Z 4 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'root'@'%'
2020-07-04T19:30:44.790150Z 0 [Note] mysqld (mysqld 5.7.13-log) starting as process 1 ...
2020-07-04T19:30:44.802834Z 0 [Note] InnoDB: PUNCH HOLE support available
2020-07-04T19:30:44.802910Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-07-04T19:30:44.802923Z 0 [Note] InnoDB: Uses event mutexes
2020-07-04T19:30:44.802937Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2020-07-04T19:30:44.802954Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.8
2020-07-04T19:30:44.802972Z 0 [Note] InnoDB: Using Linux native AIO
2020-07-04T19:30:44.803689Z 0 [Note] InnoDB: Number of pools: 1
2020-07-04T19:30:44.803895Z 0 [Note] InnoDB: Using CPU crc32 instructions
2020-07-04T19:30:44.808248Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2020-07-04T19:30:44.830811Z 0 [Note] InnoDB: Completed initialization of buffer pool
2020-07-04T19:30:44.840993Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-07-04T19:30:44.859605Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2020-07-04T19:30:44.861798Z 0 [Note] InnoDB: Log scan progressed past the checkpoint lsn 2526017
2020-07-04T19:30:45.219047Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 7768576
2020-07-04T19:30:45.645088Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 3574272
2020-07-04T19:30:46.040926Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 8817152
2020-07-04T19:30:46.288539Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 12129251
2020-07-04T19:30:46.291556Z 0 [Note] InnoDB: Database was not shutdown normally!
2020-07-04T19:30:46.291623Z 0 [Note] InnoDB: Starting crash recovery.
2020-07-04T19:30:46.311235Z 0 [Note] InnoDB: Transaction 1291 was in the XA prepared state.
2020-07-04T19:30:46.369621Z 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 0 row operations to undo
2020-07-04T19:30:46.369695Z 0 [Note] InnoDB: Trx id counter is 1792
2020-07-04T19:30:46.369970Z 0 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
2020-07-04T19:30:47.649110Z 0 [Note] InnoDB: Apply batch completed
2020-07-04T19:30:47.649151Z 0 [Note] InnoDB: Last MySQL binlog file position 0 29965, file name mysql-bin.000001
2020-07-04T19:30:47.751850Z 0 [Note] InnoDB: Starting in background the rollback of uncommitted transactions
2020-07-04T19:30:47.751924Z 0 [Note] InnoDB: Rollback of non-prepared transactions completed
2020-07-04T19:30:47.757272Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2020-07-04T19:30:47.757417Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-07-04T19:30:47.757503Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-07-04T19:30:47.817258Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-07-04T19:30:47.819594Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2020-07-04T19:30:47.819704Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2020-07-04T19:30:47.821532Z 0 [Note] InnoDB: Waiting for purge to start
2020-07-04T19:30:47.871815Z 0 [Note] InnoDB: 5.7.13 started; log sequence number 12129251
2020-07-04T19:30:47.872241Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2020-07-04T19:30:47.875717Z 0 [Note] Plugin 'FEDERATED' is disabled.
2020-07-04T19:30:47.877282Z 0 [Note] Recovering after a crash using mysql-bin
2020-07-04T19:30:47.890851Z 0 [Note] InnoDB: Buffer pool(s) load completed at 200704 19:30:47
2020-07-04T19:30:48.037185Z 0 [Note] Starting crash recovery...
2020-07-04T19:30:48.038417Z 0 [Note] InnoDB: Starting recovery for XA transactions...
2020-07-04T19:30:48.038479Z 0 [Note] InnoDB: Transaction 1291 in prepared state after recovery
2020-07-04T19:30:48.038501Z 0 [Note] InnoDB: Transaction contains changes to 134650 rows
2020-07-04T19:30:48.038522Z 0 [Note] InnoDB: 1 transactions in prepared state after recovery
2020-07-04T19:30:48.038550Z 0 [Note] Found 1 prepared transaction(s) in InnoDB
2020-07-04T19:30:48.052945Z 0 [Note] Crash recovery finished.
2020-07-04T19:30:48.185585Z 0 [Warning] Failed to set up SSL because of the following SSL library error: SSL context is not usable without certificate and private key
2020-07-04T19:30:48.185642Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
2020-07-04T19:30:48.186030Z 0 [Note] IPv6 is available.
2020-07-04T19:30:48.186080Z 0 [Note] - '::' resolves to '::';
2020-07-04T19:30:48.186127Z 0 [Note] Server socket created on IP: '::'.
2020-07-04T19:30:48.190820Z 0 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:30:48.190887Z 0 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:30:48.193844Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2020-07-04T19:30:48.214651Z 0 [Note] Event Scheduler: Loaded 0 events
2020-07-04T19:30:48.218879Z 0 [Note] mysqld: ready for connections.
Version: '5.7.13-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
2020-07-04T19:30:53.147442Z 5 [Note] Start binlog_dump to master_thread_id(5) slave_server(1), pos(, 4)
2020-07-04T19:31:58.105337Z 48 [Note] Access denied for user 'ftgo_accounting_service_user'@'172.19.0.11' (using password: YES)
2020-07-04T19:31:59.206447Z 50 [Note] Access denied for user 'ftgo_restaurant_service_user'@'172.19.0.12' (using password: YES)
2020-07-04T19:31:59.298449Z 51 [Note] Access denied for user 'ftgo_consumer_service_user'@'172.19.0.13' (using password: YES)
2020-07-04T19:31:59.457779Z 53 [Note] Access denied for user 'ftgo_accounting_service_user'@'172.19.0.11' (using password: YES)
2020-07-04T19:32:00.490786Z 54 [Note] Access denied for user 'ftgo_kitchen_service_user'@'172.19.0.14' (using password: YES)
2020-07-04T19:32:00.542109Z 55 [Note] Access denied for user 'ftgo_restaurant_service_user'@'172.19.0.12' (using password: YES)
2020-07-04T19:32:00.871701Z 57 [Note] Access denied for user 'ftgo_consumer_service_user'@'172.19.0.13' (using password: YES)
2020-07-04T19:32:01.833312Z 58 [Note] Access denied for user 'ftgo_kitchen_service_user'@'172.19.0.14' (using password: YES)
2020-07-04T19:32:05.628886Z 62 [Note] Access denied for user 'ftgo_consumer_service_user'@'172.19.0.13' (using password: YES)
2020-07-04T19:32:06.337254Z 63 [Note] Access denied for user 'ftgo_accounting_service_user'@'172.19.0.11' (using password: YES)
2020-07-04T19:32:07.604991Z 65 [Note] Access denied for user 'ftgo_restaurant_service_user'@'172.19.0.12' (using password: YES)
2020-07-04T19:32:08.630495Z 67 [Note] Access denied for user 'ftgo_order_service_user'@'172.19.0.10' (using password: YES)
2020-07-04T19:32:08.677029Z 68 [Note] Access denied for user 'ftgo_kitchen_service_user'@'172.19.0.14' (using password: YES)
2020-07-04T19:32:09.763592Z 69 [Note] Access denied for user 'ftgo_order_service_user'@'172.19.0.10' (using password: YES)
2020-07-04T19:32:12.335800Z 72 [Note] Access denied for user 'ftgo_order_service_user'@'172.19.0.10' (using password: YES)
Thanks for sending. For some strange reason the DB is not initializing because of this error:
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'root'@'%'
I am not sure why this error is occurring since the DB initialization scripts do not attempt to create this user.
To ensure a clean state, please execute this command:
./gradlew :composeDown :composeUp -P startedServices=mysql
and then send the log (please put the log in a https://gist.github.com/) .
Creating the root user happens here: https://github.com/docker-library/mysql/blob/master/5.7/docker-entrypoint.sh#L236 Unclear how this could be executed more than once.
Thanks for the logs.
The CDC cannot connect to the database.
This is the problem:
502768da80f6 ftgo-application_mysql "docker-entrypoint.s…" 2 minutes ago Exited (1) About a minute ago ftgo-application_mysql_1
What's the output of
docker logs ftgo-application_mysql_1
?
A long time ago, I had the same problem.
This gave me a tip. I started the MySQL at Mac. Let me stop the MySQL database, then try it again.
@vikramsisodia27 If you submit logs as code block, it is more readable.
Put the logs at where the foo
is.
This is the document about MarkDown: GitHub Flavored Markdown Spec
I restart my Mac. And close the most application. I stopped the MySQL.
The memory was more 5G. (My machine has 16G memory.)
The ftgo-application_cdc-service_1 was Killed again.
The latest log:
Thanks for sending. For some strange reason the DB is not initializing because of this error:
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'root'@'%'
I am not sure why this error is occurring since the DB initialization scripts do not attempt to create this user.
To ensure a clean state, please execute this command:
./gradlew :composeDown :composeUp -P startedServices=mysql
and then send the log (please put the log in a https://gist.github.com/) .
Thanks for sending. For some strange reason the DB is not initializing because of this error:
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'root'@'%'
I am not sure why this error is occurring since the DB initialization scripts do not attempt to create this user.
To ensure a clean state, please execute this command:
./gradlew :composeDown :composeUp -P startedServices=mysql
and then send the log (please put the log in a https://gist.github.com/) .
Thanks Chris.
Please find logs for ./gradlew :composeDown :composeUp -P startedServices=mysql @
https://gist.github.com/vikramsisodia27/5d176203c449cf3991ad05d2b9ca5df5
The kafka did not connect to the ZooKeeper server.
@vikramsisodia27 It looks like you have DOCKER_HOST_IP set to 192.168.1.5
. Setting DOCKER_HOST_IP is often optional/harmful depending on your environment. If you are using Docker for Mac/Windows you should leave it unset.
@vikramsisodia27 It looks like you have DOCKER_HOST_IP set to
192.168.1.5
. Setting DOCKER_HOST_IP is often optional/harmful depending on your environment. If you are using Docker for Mac/Windows you should leave it unset.
My fault - I changed MYSQL_ROOT_PASSWORD in docker-compose.yml along with docker port
@cer Appreciate your kindness and dedication to resolve novice microservice developer's workspace issue
@diguage Thanks for letting me know about code block, its nice
@cer @vikramsisodia27
The problem may really be caused by the lack of resources.
I change the configuration of Docker:
Then it is OK:
@cer @vikramsisodia27
The problem may really be caused by the lack of resources.
I change the configuration of Docker:
Then it is OK:
This question of cdc confused me whole day, luckily I get this issure opened by you. @cer could you limit the use of source of cdc, without Mac, on my low configuration Windowds PC, this Ftgo fail to run. Thank you all
Hi all,
I can comfirm that increasing the default resources, specifically the available RAM from 2 to 6 GB (which worked in my case, even on a poor MacBook Pro 13 with only 8 GB (!) RAM) in the Preferences of Docker Desktop for Mac actually works.
For more details refer to my other comment. And yes, it took over 10 minutes for all services to launch (in my case) - so be patient :)
Waiting for cdc-service_1 to become healthy (it's unhealthy) - using ./gradlew :composeUp