Closed pulnara closed 3 years ago
I haven't ran this in a while.
Are there any logs for the ftgo-accounting-service
pod?
No, kubectl logs
command (ran the script once again just now, which is why the pod IDs are different) returns nothing:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
ftgo-accounting-service-55bd94d788-cg6jp 0/1 Running 2 6m11s
ftgo-api-gateway-79948fdfbc-g5fgt 0/1 Running 2 6m10s
ftgo-cdc-service-58fcf454b5-9qvw6 0/1 Running 2 6m11s
ftgo-consumer-service-59fd7cd94d-bpbng 0/1 Running 2 6m9s
ftgo-dynamodb-local-0 1/1 Running 0 7m36s
ftgo-dynamodb-local-init-chzrw 0/1 Error 0 7m36s
ftgo-dynamodb-local-init-nchmp 1/1 Running 0 4m
ftgo-kafka-0 1/1 Running 0 7m35s
ftgo-kitchen-service-868476cb5c-h7kg5 0/1 Running 2 6m8s
ftgo-mysql-0 1/1 Running 0 7m34s
ftgo-order-history-service-b84cb6f49-f4877 0/1 Running 2 6m7s
ftgo-order-service-58b776fc7c-9t6zl 0/1 Running 2 6m5s
ftgo-restaurant-service-5fdf567678-t5sk2 0/1 Running 1 6m4s
ftgo-zookeeper-0 1/1 Running 0 7m33s
Hmmm... what about kubectl logs -p
?
It returns the following:
$ kubectl logs -p ftgo-accounting-service-55bd94d788-cg6jp
2021-04-26 16:02:49.885 INFO [,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration' of type [org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.2.1.RELEASE)
2021-04-26 16:02:56.472 INFO [ftgo-accounting-service,,,] 8 --- [ main] n.c.f.a.main.AccountingServiceMain : No active profile set, falling back to default profiles: default
2021-04-26 16:03:55.502 INFO [ftgo-accounting-service,,,] 8 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data repositories in DEFAULT mode.
2021-04-26 16:03:56.347 INFO [ftgo-accounting-service,,,] 8 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 552ms. Found 0 repository interfaces.
2021-04-26 16:04:07.025 INFO [ftgo-accounting-service,,,] 8 --- [ main] o.s.cloud.context.scope.GenericScope : BeanFactory id=bb8bd870-2d57-3dd4-a7e9-669745dbf85a
2021-04-26 16:04:16.353 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:16.938 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'io.eventuate.javaclient.spring.EventuateJavaClientDomainConfiguration' of type [io.eventuate.javaclient.spring.EventuateJavaClientDomainConfiguration$$EnhancerBySpringCGLIB$$f16b79d7] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:17.258 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'io.eventuate.javaclient.spring.common.EventuateCommonConfiguration' of type [io.eventuate.javaclient.spring.common.EventuateCommonConfiguration$$EnhancerBySpringCGLIB$$1b424602] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:17.370 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'io.eventuate.local.java.spring.common.jdbckafkastore.EventuateLocalConfiguration' of type [io.eventuate.local.java.spring.common.jdbckafkastore.EventuateLocalConfiguration$$EnhancerBySpringCGLIB$$a5e7fd80] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:17.988 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'dataSource' of type [org.springframework.aop.scope.ScopedProxyFactoryBean] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:17.996 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'dataSource' of type [com.zaxxer.hikari.HikariDataSource$$EnhancerBySpringCGLIB$$15533138] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:18.229 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'spring.datasource-org.springframework.boot.autoconfigure.jdbc.DataSourceProperties' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:18.374 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.jdbc.DataSourceInitializerInvoker' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceInitializerInvoker] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:18.486 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'spring.jpa-org.springframework.boot.autoconfigure.orm.jpa.JpaProperties' of type [org.springframework.boot.autoconfigure.orm.jpa.JpaProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:18.535 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'spring.jpa.hibernate-org.springframework.boot.autoconfigure.orm.jpa.HibernateProperties' of type [org.springframework.boot.autoconfigure.orm.jpa.HibernateProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:18.611 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.jdbc.metadata.DataSourcePoolMetadataProvidersConfiguration$HikariPoolDataSourceMetadataProviderConfiguration' of type [org.springframework.boot.autoconfigure.jdbc.metadata.DataSourcePoolMetadataProvidersConfiguration$HikariPoolDataSourceMetadataProviderConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:18.676 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'hikariPoolDataSourceMetadataProvider' of type [org.springframework.boot.autoconfigure.jdbc.metadata.DataSourcePoolMetadataProvidersConfiguration$HikariPoolDataSourceMetadataProviderConfiguration$$Lambda$255/1732238286] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:19.019 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration' of type [org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:19.218 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.transaction.TransactionAutoConfiguration' of type [org.springframework.boot.autoconfigure.transaction.TransactionAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:19.251 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'spring.transaction-org.springframework.boot.autoconfigure.transaction.TransactionProperties' of type [org.springframework.boot.autoconfigure.transaction.TransactionProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:19.262 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'platformTransactionManagerCustomizers' of type [org.springframework.boot.autoconfigure.transaction.TransactionManagerCustomizers] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:20.637 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'jpaVendorAdapter' of type [org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:20.711 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'entityManagerFactoryBuilder' of type [org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-04-26 16:04:21.299 INFO [ftgo-accounting-service,,,] 8 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.jdbc.DataSourceConfiguration$Hikari' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceConfiguration$Hikari] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
It seems that introducing lazy initialization might help. I'm not very familiar with Spring beans yet, so I'm not sure where to fix it though.
It seems that introducing lazy initialization might help. I'm not very familiar with Spring beans yet, so I'm not sure where to fix it though.
It's probably best to assume that the service code works at the application level. The probably is mostly likely with where it's running (i.e. K8s) and how it has been configured to run (e.g. k8s yaml)..
That log is useful. In particular it doesn't show any errors, which is interesting.
A couple of thoughts:
What's the configured memory of the Docker VM? The default settings are way too low to run Java services.
Another potential problem is that starting all of the services (including CDC) in one go is very resource intensive and can potentially overload a machine. I have a vague memory of situation where due to a lack of resources the services could not start in time and were repeatedly killed. I would recommend starting services one at a time to see whether that solves the problem.
I've just tried running this on another machine - MacBook Pro with 16GB RAM (previously used MacBook Air/8 GB and Ubuntu machine with 16 GB as well). Here are my Docker resources settings:
After applying this change, I tried starting only the ftgo-application-service
(after successfully deploying all of the stateful-services
) and it didn't work either, resulting with similar logs.
What's your recommended memory/resources configuration?
After a couple more attempts, and starting minikube with the following specification:
minikube start --cpus 4 --memory 4096
I can see that the problem might lay in the service being unable to reach Kafka topic. Here is some part of the accounting service's pod logs that indicates it:
2021-04-27 21:44:07.319 INFO [ftgo-accounting-service,,,] 7 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka version: 2.3.0
2021-04-27 21:44:07.319 INFO [ftgo-accounting-service,,,] 7 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka commitId: fc1aaa116b661c8a
2021-04-27 21:44:07.319 INFO [ftgo-accounting-service,,,] 7 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka startTimeMs: 1619559847318
2021-04-27 21:44:07.324 DEBUG [ftgo-accounting-service,,,] 7 --- [ main] i.e.m.k.b.c.EventuateKafkaConsumer : Verifying Topic net.chrisrichardson.ftgo.consumerservice.domain.Consumer
2021-04-27 21:45:07.339 ERROR [ftgo-accounting-service,,,] 7 --- [ main] i.e.m.k.b.c.EventuateKafkaConsumer : Got exception:
org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
2021-04-27 21:45:07.349 ERROR [ftgo-accounting-service,,,] 7 --- [ main] i.e.m.k.b.c.EventuateKafkaConsumer : Error subscribing
java.lang.RuntimeException: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
at io.eventuate.messaging.kafka.basic.consumer.EventuateKafkaConsumer.verifyTopicExistsBeforeSubscribing(EventuateKafkaConsumer.java:73) ~[eventuate-messaging-kafka-basic-consumer-0.10.0.RELEASE.jar!/:na]
at io.eventuate.messaging.kafka.basic.consumer.EventuateKafkaConsumer.start(EventuateKafkaConsumer.java:98) ~[eventuate-messaging-kafka-basic-consumer-0.10.0.RELEASE.jar!/:na]
at io.eventuate.messaging.kafka.consumer.MessageConsumerKafkaImpl.subscribe(MessageConsumerKafkaImpl.java:55) [eventuate-messaging-kafka-consumer-0.10.0.RELEASE.jar!/:na]
at io.eventuate.tram.consumer.kafka.EventuateTramKafkaMessageConsumer.subscribe(EventuateTramKafkaMessageConsumer.java:28) [eventuate-tram-consumer-kafka-0.25.1.RELEASE.jar!/:na]
at io.eventuate.tram.consumer.common.MessageConsumerImpl.subscribe(MessageConsumerImpl.java:37) [eventuate-tram-consumer-common-0.25.1.RELEASE.jar!/:na]
at io.eventuate.tram.events.subscriber.DomainEventDispatcher.initialize(DomainEventDispatcher.java:35) [eventuate-tram-events-0.25.1.RELEASE.jar!/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_252]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_252]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_252]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_252]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389) [spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333) [spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157) [spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:416) [spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1795) [spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:595) [spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) [spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) [spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) [spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) [spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879) ~[spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at net.chrisrichardson.ftgo.accountingservice.main.AccountingServiceMain.main(AccountingServiceMain.java:24) ~[classes!/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_252]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_252]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_252]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_252]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) ~[service.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) ~[service.jar:na]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) ~[service.jar:na]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) ~[service.jar:na]
Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
Please find the full log attached here: accounting-service-logs.txt
I believe that the kafka service doesn't work properly. I'm attaching its logs as well: kafka-pod-logs.txt
Finally an error message!
The Kafka log indicates that the DNS name ftgo-zookeeper
cannot be resolved. Is there a K8s ftgo-zookeeper service?
PS. I am surprised that you need to run minikube start --cpus 4 --memory 4096
- why not the k8s built into Docker?
Finally an error message!
The Kafka log indicates that the DNS name
ftgo-zookeeper
cannot be resolved. Is there a K8s ftgo-zookeeper service?
It's up and running. Plese find the full pod's log here: zookeeper-pod-logs.txt
PS. I am surprised that you need to run
minikube start --cpus 4 --memory 4096
- why not the k8s built into Docker?
I'm just used to using minikube
:) I definitely should try the built-in K8s then.
Thank you very much for your assistance, I'm still learning :)
I'm just used to using
minikube
:) I definitely should try the built-in K8s then.
Presumably, that's why increasing the Docker VM size did not have an effect.
It's up and running. Plese find the full pod's log here: zookeeper-pod-logs.txt
This indicates that there is a pod. It tell you whether there is a Service defined - https://github.com/microservices-patterns/ftgo-application/blob/a835e23bb0f3bc92dd712ff48a1510496ecb10fa/deployment/kubernetes/stateful-services/ftgo-zookeeper-deployment.yml#L1 - that configures the DNS name
BTW I am bit confused about that's happening. I read more of the Kafka log. It seems to evaluating connect to Kafka. But then it (sometimes) has problems connecting to itself via its advertised name ftgo-kafka:9092
when trying to create the topic net.chrisrichardson.ftgo.consumerservice.domain.Consumer
(the problem in the accounting service)
[2021-04-27 21:52:10,203] WARN [RequestSendThread controllerId=1001] Controller 1001's connection to broker ftgo-kafka:9092 (id: 1001 rack: null) was unsuccessful (kafka.controller.RequestSendThread)
java.net.SocketTimeoutException: Failed to connect within 30000 ms
at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:283)
at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:236)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
I've check and it seems that topic is created but I also cannot find the reason of the SocketTimeoutException
Okay - I think I know what is the reason of this problem.
After you've drawn my attention to the DNS name issue, I found some discussion on StackOverflow which indicated that Minikube doesn't usually work well with names, preferring IPs. I decided to use the K8s built-in into Docker, and it worked! Even on low resources settings.
Example screenshot after I deployed 2 of the services:
I will switch to Docker built-in K8s or GKS then. Since the app works like a charm after I've updated the outdated API in K8s objects, I took the liberty of creating a PR to integrate those changes: https://github.com/microservices-patterns/ftgo-application/pull/131
@cer thank you very much for your patience and help in debugging! 😄
I'm currently trying to run this application in Kubernetes, but with no luck. I fixed a couple of problems with the deployments in my forked version: https://github.com/pulnara/ftgo-application
I'm using a Minikube Kubernetes cluster and the
kubernetes-deploy-all.sh
script. After executing it, pods responsible for Kafka, MySQL etc. are successfully running, but the ones for business logic (like accounting service, consumer service etc.) not. After a while (and multiple restarts) they're eventually going into theCrashLoopBackOff
status, since all the time they don't meet the liveness probe - getting "connection refused" for the Spring Boot's/actuator/health
endpoint.This is my result of the
kubectl get pods
command:This are events from the ftgo-accounting-service pod's description: Events:
Did anyone happen to have a similar problem? I tried multiple times and on 2 different OS (Ubuntu, Mac OS), and getting the same error on both. I have no idea how to fix it.