brettwooldridge / HikariCP

光 HikariCP・A solid, high-performance, JDBC connection pool at last.
Apache License 2.0
20k stars 2.93k forks source link

Hikari Drop /Lost Connections - DB connections are lost (goes missing )and Get jdbc Connection Error #1626

Closed dheerajprasad closed 4 years ago

dheerajprasad commented 4 years ago

Im using Spring Boot With Hikari , Please find the connection details below. As when the application runs. The connection depletes and becomes zero spring.datasource.username=test spring.datasource.password=test spring.datasource.hikari.maximum-pool-size = 100 spring.datasource.hikari.idleTimeout=180000 spring.datasource.hikari.maxLifetime=300000 spring.datasource.hikari.minimum-idle= 10 As when application runs , Connection pool reduces and becomes Zero and we get DB exception. Find the logs below. UT 2020-07-23 16:52:23.420 DEBUG [Test-service,,,] 13 --- [HikariPool-1 housekeeper] om.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Before cleanup stats (total=0, active=0, idle=0, waiting=2)

OUT 2020-07-23 16:51:53.419 DEBUG [Test-service,,,] 13 --- [HikariPool-1 housekeeper] 
com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Before cleanup stats (total=2, active=0, 
idle=2, waiting=0)

OUT 2020-07-23 16:51:23.418 DEBUG [Test-service,,,] 13 --- [HikariPool-1 housekeeper] 
com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Before cleanup stats (total=4, active=1, 
idle=3, waiting=0)

OUT 2020-07-23 16:50:53.418 DEBUG [Test-service,,,] 13 --- [HikariPool-1 housekeeper] 
om.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Before cleanup stats 
(total=4, active=0, 
idle=4, waiting=0)

OUT 2020-07-23 16:50:23.417 DEBUG [Test-service,,,] 13 --- [HikariPool-1 housekeeper]     
com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Before cleanup stats 
(total=4, active=0,       idle=4, 
waiting=0)
Thu, Jul 23, 2020 16:49:53.417
[APP/PROC/WEB/0]
OUT 2020-07-23 16:49:53.416 DEBUG [Test-service,,,] 13 --- [HikariPool-1 
sekeeper] 
com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Before cleanup stats (total=4, active=0,     id le=4, waiting=0)
 Thu, Jul 23, 2020 16:49:23.416
[APP/PROC/WEB/0]
OUT 2020-07-23 16:49:23.416 DEBUG [Test-service,,,] 13 --- [HikariPool-1 housekeeper] 
com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Before cleanup stats (total=4, active=0,   idle=4, waiting=0)
Thu, Jul 23, 2020 16:48:53.416
[APP/PROC/WEB/0]
OUT 2020-07-23 16:48:53.415 DEBUG [Test-service,,,] 13 --- [HikariPool-1 housekeeper] 
com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Before cleanup stats (total=5, active=0,     idle=5, waiting=0)
Thu, Jul 23, 2020 16:48:23.415
[APP/PROC/WEB/0]
dheerajprasad commented 4 years ago

Attaching the logs.

com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before cleanup stats (total=2, active=0, idle=2, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:36.911213144Z OUT 2020-07-24 14:22:36.910 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After cleanup  stats (total=2, active=0, idle=2, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:41.725695633Z OUT 2020-07-24 14:22:41.725  INFO [debitcard-service,de061f504917ad8c,0251615909131852,false] 14 --- [http-nio-8080-exec-5] c.dbs.mdhk.common.filter.LoggingFilter   : Transaction Id in request:: null 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:41.725718196Z OUT 2020-07-24 14:22:41.725  INFO [debitcard-service,de061f504917ad8c,0251615909131852,false] 14 --- [http-nio-8080-exec-5] c.dbs.mdhk.common.filter.LoggingFilter   : Session Id in request:: null 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:41.725721251Z OUT 2020-07-24 14:22:41.725  INFO [debitcard-service,de061f504917ad8c,0251615909131852,false] 14 --- [http-nio-8080-exec-5] c.dbs.mdhk.common.filter.LoggingFilter   : URI:: /debitcard/cif/7574571 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:41.726540962Z OUT 2020-07-24 14:22:41.726  INFO [debitcard-service,de061f504917ad8c,0251615909131852,false] 14 --- [http-nio-8080-exec-5] PERFORMANCE                              : {"timestamp_log":"2020-07-24 14:22:41,726","type":"debitcard-service","serviceID":"GET/debitcard/cif/{cif}","clientIp":"10.92.227.50","statusCode":"","api_type":"request"} 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:41.774469434Z OUT 2020-07-24 14:22:41.774  INFO [debitcard-service,de061f504917ad8c,0251615909131852,false] 14 --- [http-nio-8080-exec-5] PERFORMANCE                              : {"timestamp_log":"2020-07-24 14:22:41,774","type":"debitcard-service","serviceID":"GET/debitcard/cif/{cif}","clientIp":"10.92.227.50","statusCode":"200","api_type":"response"} 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:41.774590632Z OUT 2020-07-24 14:22:41.774  INFO [debitcard-service,de061f504917ad8c,0251615909131852,false] 14 --- [http-nio-8080-exec-5] c.dbs.mdhk.common.filter.LoggingFilter   : Request body::  
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:41.774596002Z OUT 2020-07-24 14:22:41.774  INFO [debitcard-service,de061f504917ad8c,0251615909131852,false] 14 --- [http-nio-8080-exec-5] c.dbs.mdhk.common.filter.LoggingFilter   : Response body::  
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:55.859996375Z OUT 2020-07-24 14:22:55.859  INFO [debitcard-service,516cb35b72488400,56c5ed26c7b186ea,false] 14 --- [http-nio-8080-exec-6] c.dbs.mdhk.common.filter.LoggingFilter   : Transaction Id in request:: null 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:55.860043243Z OUT 2020-07-24 14:22:55.859  INFO [debitcard-service,516cb35b72488400,56c5ed26c7b186ea,false] 14 --- [http-nio-8080-exec-6] c.dbs.mdhk.common.filter.LoggingFilter   : Session Id in request:: null 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:55.860049124Z OUT 2020-07-24 14:22:55.859  INFO [debitcard-service,516cb35b72488400,56c5ed26c7b186ea,false] 14 --- [http-nio-8080-exec-6] c.dbs.mdhk.common.filter.LoggingFilter   : URI:: /debitcard/cif/7574571 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:55.861670791Z OUT 2020-07-24 14:22:55.861  INFO [debitcard-service,516cb35b72488400,56c5ed26c7b186ea,false] 14 --- [http-nio-8080-exec-6] PERFORMANCE                              : {"timestamp_log":"2020-07-24 14:22:55,861","type":"debitcard-service","serviceID":"GET/debitcard/cif/{cif}","clientIp":"10.92.227.50","statusCode":"","api_type":"request"} 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:55.869415342Z OUT 2020-07-24 14:22:55.869  INFO [debitcard-service,516cb35b72488400,56c5ed26c7b186ea,false] 14 --- [http-nio-8080-exec-6] PERFORMANCE                              : {"timestamp_log":"2020-07-24 14:22:55,869","type":"debitcard-service","serviceID":"GET/debitcard/cif/{cif}","clientIp":"10.92.227.50","statusCode":"200","api_type":"response"} 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:55.869583278Z OUT 2020-07-24 14:22:55.869  INFO [debitcard-service,516cb35b72488400,56c5ed26c7b186ea,false] 14 --- [http-nio-8080-exec-6] c.dbs.mdhk.common.filter.LoggingFilter   : Request body::  
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:22:55.86960062Z OUT 2020-07-24 14:22:55.869  INFO [debitcard-service,516cb35b72488400,56c5ed26c7b186ea,false] 14 --- [http-nio-8080-exec-6] c.dbs.mdhk.common.filter.LoggingFilter   : Response body::  
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:23:06.911233476Z OUT 2020-07-24 14:23:06.910 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before cleanup stats (total=2, active=0, idle=2, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:23:06.911284823Z OUT 2020-07-24 14:23:06.911 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After cleanup  stats (total=2, active=0, idle=2, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:23:36.911725614Z OUT 2020-07-24 14:23:36.911 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before cleanup stats (total=2, active=0, idle=2, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:23:36.911825792Z OUT 2020-07-24 14:23:36.911 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After cleanup  stats (total=2, active=0, idle=2, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:23:47.685781811Z OUT 2020-07-24 14:23:47.685 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@5ef4ba81: (connection has passed maxLifetime) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:06.923039004Z OUT 2020-07-24 14:24:06.911 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before cleanup stats (total=1, active=0, idle=1, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:06.92307962Z OUT 2020-07-24 14:24:06.912 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After cleanup  stats (total=1, active=0, idle=1, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:15.015726153Z OUT 2020-07-24 14:24:15.015  INFO [debitcard-service,c8c4061f9ac06d74,c8c4061f9ac06d74,false] 14 --- [http-nio-8080-exec-4] c.dbs.mdhk.common.filter.LoggingFilter   : Transaction Id in request:: null 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:15.015744778Z OUT 2020-07-24 14:24:15.015  INFO [debitcard-service,c8c4061f9ac06d74,c8c4061f9ac06d74,false] 14 --- [http-nio-8080-exec-4] c.dbs.mdhk.common.filter.LoggingFilter   : Session Id in request:: null 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:15.015747533Z OUT 2020-07-24 14:24:15.015  INFO [debitcard-service,c8c4061f9ac06d74,c8c4061f9ac06d74,false] 14 --- [http-nio-8080-exec-4] c.dbs.mdhk.common.filter.LoggingFilter   : URI:: /grpweb/debitcard 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:15.016576431Z OUT 2020-07-24 14:24:15.016  INFO [debitcard-service,c8c4061f9ac06d74,c8c4061f9ac06d74,false] 14 --- [http-nio-8080-exec-4] PERFORMANCE                              : {"timestamp_log":"2020-07-24 14:24:15,016","type":"debitcard-service","serviceID":"GET/grpweb/debitcard","clientIp":"10.92.227.50","statusCode":"","api_type":"request"} 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:15.03786515Z OUT 2020-07-24 14:24:15.037  INFO [debitcard-service,c8c4061f9ac06d74,c8c4061f9ac06d74,false] 14 --- [http-nio-8080-exec-4] c.d.m.d.r.DebitCardRepositoryImpl        : Existing cards for the interval: 0 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:15.038671966Z OUT 2020-07-24 14:24:15.038  INFO [debitcard-service,c8c4061f9ac06d74,c8c4061f9ac06d74,false] 14 --- [http-nio-8080-exec-4] PERFORMANCE                              : {"timestamp_log":"2020-07-24 14:24:15,038","type":"debitcard-service","serviceID":"GET/grpweb/debitcard","clientIp":"10.92.227.50","statusCode":"200","api_type":"response"} 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:15.038733371Z OUT 2020-07-24 14:24:15.038  INFO [debitcard-service,c8c4061f9ac06d74,c8c4061f9ac06d74,false] 14 --- [http-nio-8080-exec-4] c.dbs.mdhk.common.filter.LoggingFilter   : Request body::  
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:15.038761665Z OUT 2020-07-24 14:24:15.038  INFO [debitcard-service,c8c4061f9ac06d74,c8c4061f9ac06d74,false] 14 --- [http-nio-8080-exec-4] c.dbs.mdhk.common.filter.LoggingFilter   : Response body:: [] 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:36.91310387Z OUT 2020-07-24 14:24:36.912 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before cleanup stats (total=1, active=0, idle=1, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:36.91314655Z OUT 2020-07-24 14:24:36.912 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After cleanup  stats (total=1, active=0, idle=1, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:55.791482798Z OUT 2020-07-24 14:24:55.791 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@3571ea2d: (connection has passed maxLifetime) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:24:58.357555616Z OUT 2020-07-24 14:24:58.357  INFO [debitcard-service,,,] 14 --- [AsyncResolver-bootstrap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver      : Resolving eureka endpoints via configuration 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:25:06.914048268Z OUT 2020-07-24 14:25:06.913 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before cleanup stats (total=0, active=0, idle=0, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:25:06.914077302Z OUT 2020-07-24 14:25:06.913 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After cleanup  stats (total=0, active=0, idle=0, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:25:36.914141636Z OUT 2020-07-24 14:25:36.913 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before cleanup stats (total=0, active=0, idle=0, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:25:36.914156314Z OUT 2020-07-24 14:25:36.913 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After cleanup  stats (total=0, active=0, idle=0, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:26:06.91441873Z OUT 2020-07-24 14:26:06.914 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before cleanup stats (total=0, active=0, idle=0, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:26:06.914453225Z OUT 2020-07-24 14:26:06.914 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After cleanup  stats (total=0, active=0, idle=0, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:26:16.261877627Z OUT 2020-07-24 14:26:16.261  INFO [debitcard-service,c703afa1f7f32520,c703afa1f7f32520,false] 14 --- [http-nio-8080-exec-9] c.dbs.mdhk.common.filter.LoggingFilter   : Transaction Id in request:: null 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:26:16.26198535Z OUT 2020-07-24 14:26:16.261  INFO [debitcard-service,c703afa1f7f32520,c703afa1f7f32520,false] 14 --- [http-nio-8080-exec-9] c.dbs.mdhk.common.filter.LoggingFilter   : Session Id in request:: null 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:26:16.26199095Z OUT 2020-07-24 14:26:16.261  INFO [debitcard-service,c703afa1f7f32520,c703afa1f7f32520,false] 14 --- [http-nio-8080-exec-9] c.dbs.mdhk.common.filter.LoggingFilter   : URI:: /grpweb/debitcard 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:26:16.263761066Z OUT 2020-07-24 14:26:16.263  INFO [debitcard-service,c703afa1f7f32520,c703afa1f7f32520,false] 14 --- [http-nio-8080-exec-9] PERFORMANCE                              : {"timestamp_log":"2020-07-24 14:26:16,263","type":"debitcard-service","serviceID":"GET/grpweb/debitcard","clientIp":"10.92.227.50","statusCode":"","api_type":"request"} 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:26:36.91477855Z OUT 2020-07-24 14:26:36.914 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before cleanup stats (total=0, active=0, idle=0, waiting=1) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:26:36.914920507Z OUT 2020-07-24 14:26:36.914 DEBUG [debitcard-service,,,] 14 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After cleanup  stats (total=0, active=0, idle=0, waiting=1) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:26:46.267613648Z OUT 2020-07-24 14:26:46.264 DEBUG [debitcard-service,c703afa1f7f32520,c703afa1f7f32520,false] 14 --- [http-nio-8080-exec-9] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Timeout failure stats (total=0, active=0, idle=0, waiting=0) 
587cff89-dc15-4296-97d9-18630b9c5696 APP/PROC/WEB/0 2020-07-24T06:26:46.304446192Z OUT 2020-07-24 14:26:46.302 ERROR [debitcard-service,c703afa1f7f32520,c703afa1f7f32520,false] 14 --- [http-nio-8080-exec-9] c.d.m.c.e.GlobalExceptionHandler         : Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms. org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms.
 at 
dheerajprasad commented 4 years ago

I kept the leak Threshold to 10 seconds and do not see any leak error as well

dheerajprasad commented 4 years ago

Noticed the Connection Closes and does not try to reconnect again.

@brettwooldridge - Can you help on this issue pls.

9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:15:53.138237373Z OUT 2020-08-26 15:15:53.136 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@3064c725: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:15:54.183726109Z OUT 2020-08-26 15:15:54.183 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@372074c2: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:15:55.24291721Z OUT 2020-08-26 15:15:55.242 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@49e84b37: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:15:57.84557564Z OUT 2020-08-26 15:15:57.844 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@52ddf48b: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:15:58.558984032Z OUT 2020-08-26 15:15:58.558 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@3430f837: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:00.425573323Z OUT 2020-08-26 15:16:00.424 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@1f7b49cd: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:05.296276046Z OUT 2020-08-26 15:16:05.295 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@2f3455c3: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:05.904562759Z OUT 2020-08-26 15:16:05.903 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=64, active=0, idle=64, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:12.658878115Z OUT 2020-08-26 15:16:12.657 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@39e79c56: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:16.516703121Z OUT 2020-08-26 15:16:16.516 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@18f56922: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:25.329166652Z OUT 2020-08-26 15:16:25.328 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@3dd81fb2: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:30.594930477Z OUT 2020-08-26 15:16:30.594 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@174cb717: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:35.904603393Z OUT 2020-08-26 15:16:35.904 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=60, active=0, idle=60, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:39.959770441Z OUT 2020-08-26 15:16:39.959 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@1c76f73a: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:43.31860079Z OUT 2020-08-26 15:16:43.318 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@4f2ef75c: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:47.166634233Z OUT 2020-08-26 15:16:47.166 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@d40af2: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:54.153929546Z OUT 2020-08-26 15:16:54.153 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@45d69c2c: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:55.704540311Z OUT 2020-08-26 15:16:55.704 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@4058e4de: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:16:56.730518798Z OUT 2020-08-26 15:16:56.729 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@387899a3: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:17:05.904936144Z OUT 2020-08-26 15:17:05.904 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=54, active=0, idle=54, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:17:06.866835956Z OUT 2020-08-26 15:17:06.866 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@41e900c9: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:17:22.28211415Z OUT 2020-08-26 15:17:22.281 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@311e292d: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:17:35.905258505Z OUT 2020-08-26 15:17:35.904 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:18:05.905459039Z OUT 2020-08-26 15:18:05.905 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:18:09.049855243Z OUT 2020-08-26 15:18:09.049  INFO [onboarding-service,,,] 13 --- [AsyncResolver-bootstrap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver      : Resolving eureka endpoints via configuration 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:18:35.905871459Z OUT 2020-08-26 15:18:35.905 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:19:05.906499913Z OUT 2020-08-26 15:19:05.905 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:19:35.906644863Z OUT 2020-08-26 15:19:35.906 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:20:05.907802949Z OUT 2020-08-26 15:20:05.906 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:20:35.907460846Z OUT 2020-08-26 15:20:35.907 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:21:05.907565811Z OUT 2020-08-26 15:21:05.907 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:21:10.257050553Z OUT 2020-08-26 15:21:10.256  INFO [onboarding-service,a9bede342fe2a1f6,a9bede342fe2a1f6,false] 13 --- [pool-2-thread-1] c.d.mdhk.onboarding.client.TokenClient   : clearing the jwt token in cache 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:21:35.907821269Z OUT 2020-08-26 15:21:35.907 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:22:05.90849458Z OUT 2020-08-26 15:22:05.908 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:23:05.909284038Z OUT 2020-08-26 15:23:05.908 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:23:09.051289754Z OUT 2020-08-26 15:23:09.050  INFO [onboarding-service,,,] 13 --- [AsyncResolver-bootstrap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver      : Resolving eureka endpoints via configuration 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:23:35.909533024Z OUT 2020-08-26 15:23:35.909 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:24:05.909909699Z OUT 2020-08-26 15:24:05.909 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:24:35.910240288Z OUT 2020-08-26 15:24:35.909 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:25:05.91051932Z OUT 2020-08-26 15:25:05.910 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:25:35.910882059Z OUT 2020-08-26 15:25:35.910 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:26:05.911198482Z OUT 2020-08-26 15:26:05.910 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:26:35.911491551Z OUT 2020-08-26 15:26:35.911 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:27:05.913028808Z OUT 2020-08-26 15:27:05.911 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:27:10.257025155Z OUT 2020-08-26 15:27:10.256  INFO [onboarding-service,20b28f88f7e32047,20b28f88f7e32047,false] 13 --- [pool-2-thread-1] c.d.mdhk.onboarding.client.TokenClient   : clearing the jwt token in cache 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:27:35.912222839Z OUT 2020-08-26 15:27:35.911 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:28:05.912438363Z OUT 2020-08-26 15:28:05.912 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:28:09.051568786Z OUT 2020-08-26 15:28:09.051  INFO [onboarding-service,,,] 13 --- [AsyncResolver-bootstrap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver      : Resolving eureka endpoints via configuration 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:28:35.912705483Z OUT 2020-08-26 15:28:35.912 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:29:05.913261113Z OUT 2020-08-26 15:29:05.912 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:29:35.913321416Z OUT 2020-08-26 15:29:35.912 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:30:05.913554713Z OUT 2020-08-26 15:30:05.913 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:30:35.914357326Z OUT 2020-08-26 15:30:35.913 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:31:05.914741805Z OUT 2020-08-26 15:31:05.914 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:31:35.915867613Z OUT 2020-08-26 15:31:35.914 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:32:05.915375061Z OUT 2020-08-26 15:32:05.914 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:32:35.916972522Z OUT 2020-08-26 15:32:35.916 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:33:05.916856434Z OUT 2020-08-26 15:33:05.916 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:33:09.05257022Z OUT 2020-08-26 15:33:09.052  INFO [onboarding-service,,,] 13 --- [AsyncResolver-bootstrap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver      : Resolving eureka endpoints via configuration 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:33:10.257703264Z OUT 2020-08-26 15:33:10.257  INFO [onboarding-service,ca0b7e0056dd683c,ca0b7e0056dd683c,false] 13 --- [pool-2-thread-1] c.d.mdhk.onboarding.client.TokenClient   : clearing the jwt token in cache 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:33:35.918154184Z OUT 2020-08-26 15:33:35.917 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:34:05.918753906Z OUT 2020-08-26 15:34:05.918 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:34:35.919096608Z OUT 2020-08-26 15:34:35.918 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:35:05.919651998Z OUT 2020-08-26 15:35:05.919 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:35:35.919798773Z OUT 2020-08-26 15:35:35.919 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:36:05.920068628Z OUT 2020-08-26 15:36:05.919 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:36:35.920568513Z OUT 2020-08-26 15:36:35.920 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:37:05.921705011Z OUT 2020-08-26 15:37:05.921 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:37:35.921793777Z OUT 2020-08-26 15:37:35.921 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:38:09.0530646Z OUT 2020-08-26 15:38:09.052  INFO [onboarding-service,,,] 13 --- [AsyncResolver-bootstrap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver      : Resolving eureka endpoints via configuration 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:38:35.923364888Z OUT 2020-08-26 15:38:35.923 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:39:05.923742428Z OUT 2020-08-26 15:39:05.923 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:39:10.257317697Z OUT 2020-08-26 15:39:10.257  INFO [onboarding-service,0e7b53ee6dd72bb4,0e7b53ee6dd72bb4,false] 13 --- [pool-2-thread-1] c.d.mdhk.onboarding.client.TokenClient   : clearing the jwt token in cache 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:39:35.923973532Z OUT 2020-08-26 15:39:35.923 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:40:05.924171686Z OUT 2020-08-26 15:40:05.923 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:40:35.924630527Z OUT 2020-08-26 15:40:35.924 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:41:05.924666808Z OUT 2020-08-26 15:41:05.924 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=52, active=0, idle=52, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:41:29.268179487Z OUT 2020-08-26 15:41:29.267 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@21930d20: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:41:35.925050518Z OUT 2020-08-26 15:41:35.924 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=50, active=0, idle=50, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:41:40.968971237Z OUT 2020-08-26 15:41:40.968 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@527be81e: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:41:43.997821887Z OUT 2020-08-26 15:41:43.997 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@59827a17: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:41:49.27956494Z OUT 2020-08-26 15:41:49.279 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@513da9e8: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:41:51.023756781Z OUT 2020-08-26 15:41:51.023 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@55023790: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:41:53.563808023Z OUT 2020-08-26 15:41:53.563 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@30f7b965: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:42:05.925527323Z OUT 2020-08-26 15:42:05.925 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=45, active=0, idle=45, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:42:20.971448184Z OUT 2020-08-26 15:42:20.970 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@5bc5c298: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:42:21.633693824Z OUT 2020-08-26 15:42:21.633 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@3e3040d3: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:42:28.599652171Z OUT 2020-08-26 15:42:28.599 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@23f78488: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:42:32.500071666Z OUT 2020-08-26 15:42:32.499 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@11235f4a: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:42:35.925512999Z OUT 2020-08-26 15:42:35.925 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=41, active=0, idle=41, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:42:37.120223795Z OUT 2020-08-26 15:42:37.119 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@39ce461b: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:42:41.422330003Z OUT 2020-08-26 15:42:41.421 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@4d655e7b: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:42:50.851028019Z OUT 2020-08-26 15:42:50.850 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@6302a2a6: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:00.145035518Z OUT 2020-08-26 15:43:00.144 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@6c0d6d42: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:00.847827424Z OUT 2020-08-26 15:43:00.847 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@2cf85a8d: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:05.925837539Z OUT 2020-08-26 15:43:05.925 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=36, active=0, idle=36, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:06.631849053Z OUT 2020-08-26 15:43:06.631 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@24179ac9: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:09.053596269Z OUT 2020-08-26 15:43:09.053  INFO [onboarding-service,,,] 13 --- [AsyncResolver-bootstrap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver      : Resolving eureka endpoints via configuration 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:10.574423035Z OUT 2020-08-26 15:43:10.574 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@62ca8eb3: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:11.612296638Z OUT 2020-08-26 15:43:11.611 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@a616f30: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:13.163722756Z OUT 2020-08-26 15:43:13.163 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@66f50632: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:20.93584075Z OUT 2020-08-26 15:43:20.935 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@438eaadd: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:26.830043245Z OUT 2020-08-26 15:43:26.829 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@69fac9d6: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:28.177934026Z OUT 2020-08-26 15:43:28.177 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@47b126f3: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:29.383503543Z OUT 2020-08-26 15:43:29.383 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@7aa4adf9: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:30.962501866Z OUT 2020-08-26 15:43:30.962 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@74634d05: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:33.232190636Z OUT 2020-08-26 15:43:33.231 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@7e9b2675: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:34.402019274Z OUT 2020-08-26 15:43:34.401 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@6dc550b6: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:35.926241558Z OUT 2020-08-26 15:43:35.925 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=25, active=0, idle=25, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:45.446773938Z OUT 2020-08-26 15:43:45.446 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@4976c1ad: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:50.632146453Z OUT 2020-08-26 15:43:50.631 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@5ff8e792: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:52.739695372Z OUT 2020-08-26 15:43:52.739 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@217947f5: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:53.425013001Z OUT 2020-08-26 15:43:53.424 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@36e3a3f3: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:55.855310766Z OUT 2020-08-26 15:43:55.855 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@3b99a7c2: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:57.493956068Z OUT 2020-08-26 15:43:57.493 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@ae87ae2: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:43:59.704740991Z OUT 2020-08-26 15:43:59.704 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@79bdccb3: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:01.925252796Z OUT 2020-08-26 15:44:01.924 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@cab77ce: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:02.13955353Z OUT 2020-08-26 15:44:02.139 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@5bb6588d: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:05.926493221Z OUT 2020-08-26 15:44:05.926 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=16, active=0, idle=16, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:10.011114985Z OUT 2020-08-26 15:44:10.010 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@2ee9efe7: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:11.532007434Z OUT 2020-08-26 15:44:11.531 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@557cced1: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:11.766938395Z OUT 2020-08-26 15:44:11.766 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@2de74d44: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:11.895050083Z OUT 2020-08-26 15:44:11.894 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@688f0f6c: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:13.30413487Z OUT 2020-08-26 15:44:13.303 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@184a92b4: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:16.780620616Z OUT 2020-08-26 15:44:16.780 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@2431c787: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:18.123536773Z OUT 2020-08-26 15:44:18.123 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@6ca7d0dc: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:18.459039839Z OUT 2020-08-26 15:44:18.458 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@57771456: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:21.099828545Z OUT 2020-08-26 15:44:21.099 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@fe604c9: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:21.195348842Z OUT 2020-08-26 15:44:21.194 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@17bab6d0: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:23.867926728Z OUT 2020-08-26 15:44:23.867 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@4f9c776e: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:25.243397909Z OUT 2020-08-26 15:44:25.243 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@3bb1844e: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:28.095578583Z OUT 2020-08-26 15:44:28.095 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@11166140: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:30.116023745Z OUT 2020-08-26 15:44:30.115 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@4a399051: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:35.926844582Z OUT 2020-08-26 15:44:35.926 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=2, active=0, idle=2, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:36.063231865Z OUT 2020-08-26 15:44:36.062 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@1b64eabc: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:44:37.646052695Z OUT 2020-08-26 15:44:37.645 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 connection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.mariadb.jdbc.MariaDbConnection@5b03baf2: (connection has passed maxLifetime) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:45:05.92701334Z OUT 2020-08-26 15:45:05.926 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=0, active=0, idle=0, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:45:10.257073317Z OUT 2020-08-26 15:45:10.256  INFO [onboarding-service,48ed0d255f031728,48ed0d255f031728,false] 13 --- [pool-2-thread-1] c.d.mdhk.onboarding.client.TokenClient   : clearing the jwt token in cache 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:45:35.930540777Z OUT 2020-08-26 15:45:35.927 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=0, active=0, idle=0, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:46:05.928140695Z OUT 2020-08-26 15:46:05.927 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=0, active=0, idle=0, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:46:35.928478821Z OUT 2020-08-26 15:46:35.927 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=0, active=0, idle=0, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:47:05.928758647Z OUT 2020-08-26 15:47:05.928 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=0, active=0, idle=0, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:47:35.928995813Z OUT 2020-08-26 15:47:35.928 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=0, active=0, idle=0, waiting=0) 
9d5e7bcc-6c81-4cef-8398-8e4d789e3638 APP/PROC/WEB/0 2020-08-26T07:48:05.929768813Z OUT 2020-08-26 15:48:05.928 DEBUG [onboarding-service,,,] 13 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=0, active=0, idle=0, waiting=0) 
brettwooldridge commented 4 years ago

If you still care about this issue, take a thread dump when the connections start trending downward. There is a Connection Adder thread that creates new connections asynchronously, and it is either stuck (blocked in the driver or the network stack), or unable to be respawned. Without a stacktrace that is contemporaneous with the timing of the issue, there's not much I can say about it.