spring-cloud / spring-cloud-dataflow

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
https://dataflow.spring.io
Apache License 2.0
1.1k stars 578 forks source link

Can not create schedules w/ current Fabric8 K8s client #5505

Open onobc opened 11 months ago

onobc commented 11 months ago
          One update on this story for someone facing this after migrating it to 2.11 in kubernetes 1.26X version . After migrating it to SCDF 2.11, the schedules page in SCDF displays the data by interacting wit openshift cron jobs api. But it wont allow you to create schedule from scdf. For this to work, we need to update the kubernetes-client to latest version . In my case, i updated it to 
io.fabric8 kubernetes-client 6.9.0

Originally posted by @sampathsri in https://github.com/spring-cloud/spring-cloud-dataflow/issues/5416#issuecomment-1757768330

corneil commented 11 months ago

We need to determine which version of k8s is not supported by fabric8 6.9.x

sampathsri commented 10 months ago

It was not working with Kubernetes version - v1.26.4

onobc commented 10 months ago

HI @sampathsri , Do you see any exception in the SCDF log when attempting to create the schedule when using the elder client? If so, could you please share the stacktrace (or anything related in the logs at that time)?

Thanks

sampathsri commented 10 months ago

Please find attached the logs scdf-server-6c6f676fb6-jpp6x-scdf-server.log

onobc commented 10 months ago

Here is the exception from the above log file:

2023-11-03 09:37:59.388  INFO 1 --- [nio-8080-exec-7] o.s.c.d.s.k.DefaultContainerFactory      : Using Docker entry point style: exec
2023-11-03 09:37:59.490 ERROR 1 --- [nio-8080-exec-7] o.s.c.d.s.c.RestControllerAdvice         : Caught exception while handling a request

org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.NoSuchMethodError: io.fabric8.kubernetes.api.model.Lifecycle.getPostStart()Lio/fabric8/kubernetes/api/model/LifecycleHandler;
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1087)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:555)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.ForwardedHeaderFilter.doFilterInternal(ForwardedHeaderFilter.java:156)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:768)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:926)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1790)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoSuchMethodError: io.fabric8.kubernetes.api.model.Lifecycle.getPostStart()Lio/fabric8/kubernetes/api/model/LifecycleHandler;
    at org.springframework.cloud.deployer.spi.kubernetes.AbstractKubernetesDeployer.createPodSpec(AbstractKubernetesDeployer.java:236)
    at org.springframework.cloud.deployer.spi.kubernetes.KubernetesScheduler.createCronJob(KubernetesScheduler.java:239)
    at org.springframework.cloud.deployer.spi.kubernetes.KubernetesScheduler.schedule(KubernetesScheduler.java:81)
    at org.springframework.cloud.dataflow.server.service.impl.DefaultSchedulerService.schedule(DefaultSchedulerService.java:325)
    at org.springframework.cloud.dataflow.server.controller.TaskSchedulerController.save(TaskSchedulerController.java:169)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072)
    ... 48 common frames omitted
gpehh commented 7 months ago

Kubernetes-client 6.9.0 version for SCDF 2.11.0 and kubernetes 1.26 did not work for me.

When I start a task, "Caused by: java.lang.NoSuchMethodError: io.fabric8.kubernetes.api.model.Lifecycle.getPostStart()" I get an error. Can you tell me the compatible dependencies? How did you run it?

Would it be enough to add it as a mavenBom? @sampathsri mavenBom 'io.fabric8:kubernetes-client-bom:6.9.0'

What is your Spring cloud version?

2.11.0 Scdf Version with; io.fabric8:kubernetes-client:5.10.2 kubernetes.model-****:5.10.2

for these versions: 2.11.0 scdf and OpenShift 4.13, kubernetes 1.26 What versions should be the kubernetes-client dependencies ? Can we detect the compatible version? And how do I update the versions for a temporary and quick solution?

@sampathsri @onobc @cppwfs @corneil

Thank you for your supports.

onobc commented 7 months ago

Hi @gpehh , I just noticed this issue. I replied to your original comment here. TL;DR - will have to wait for SCDF 3.0.0. Dataflow 2.11.x will remain on the 5.9.x Fabric8 - sorry.