search

kubesphere / ks-devops

This is a cloud-native application that focuses on the DevOps area.
https://kubesphere.io/devops/
Apache License 2.0
157 stars 89 forks source link

When starting or creating a pipeline, the kubesphere alarm prompt 404 #1038

Open poet123 opened 6 months ago

poet123 commented 6 months ago

What is version of KubeSphere DevOps has the issue?

devops-apiserver:ks-v3.4.1 ks-jenkins:v3.4.0-2.319.3-1

How did you install the Kubernetes? Or what is the Kubernetes distribution?

My kubesphere has been upgraded to 3.4.1 after installing 3.3.1

What happened?

Today, a colleague gave feedback that the assembly line could not run. I checked the logs on devops and found that the certificate had expired. Then, I tried to update the certificate for k8s (kubeadm certificates renew all), but it did not take effect. Then, I tried to upgrade kubeshee, but it still couldn't solve the problem. I went to the developer community to check（ https://ask.kubesphere.io/forum/d/23239-kubesphere-jing-xiang-gou-jian-qi-s2ifu-wu-zheng-shu-guo-qi-wen-ti ） Fixed the issue of certificate expiration, but when operating on the watermark, it prompted the following message： image Then I checked the logs of devops-apiserver and kept prompting the following content: image Then I checked the logs of devops Jenkins and found that the user wukai1 does not exist:

image But there is no user wukai1 in kubeshpere，

image image

Relevant log output

2024-04-07 14:40:13.390+0000 [id=143]   WARNING h.i.i.InstallUncaughtExceptionHandler#handleException: Caught unhandled exception with ID 6d1c45c0-a747-44b8-b37a-0074f07ab25e

 org.acegisecurity.userdetails.UsernameNotFoundException: User wukai1 not found in directory.

    at org.acegisecurity.ldap.search.FilterBasedLdapUserSearch.searchForUser(FilterBasedLdapUserSearch.java:126)

    at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:1320)

    at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:1273)

    at org.acegisecurity.userdetails.UserDetailsService.lambda$toSpring$1(UserDetailsService.java:52)

 Caused: org.springframework.security.core.userdetails.UsernameNotFoundException

    at org.acegisecurity.userdetails.UsernameNotFoundException.toSpring(UsernameNotFoundException.java:51)

    at org.acegisecurity.userdetails.UsernameNotFoundException.toSpring(UsernameNotFoundException.java:34)

    at org.acegisecurity.userdetails.UserDetailsService.lambda$toSpring$1(UserDetailsService.java:54)

    at jenkins.security.ImpersonatingUserDetailsService2.loadUserByUsername(ImpersonatingUserDetailsService2.java:29)

    at hudson.model.User.getUserDetailsForImpersonation2(User.java:406)

    at hudson.model.User.getUserDetailsForImpersonation(User.java:429)

 Caused: org.acegisecurity.userdetails.UsernameNotFoundException: org.springframework.security.core.userdetails.UsernameNotFoundException: org.acegisecurity.userdetails.UsernameNotFoundException: User wukai1 not found in directory.; nested exception is org.springframework.security.core.userdetails.UsernameNotFoundException: org.acegisecurity.userdetails.UsernameNotFoundException: User wukai1 not found in directory.

    at org.acegisecurity.userdetails.UsernameNotFoundException.fromSpring(UsernameNotFoundException.java:58)

    at org.acegisecurity.AuthenticationException.fromSpring(AuthenticationException.java:98)

    at hudson.model.User.getUserDetailsForImpersonation(User.java:431)

    at io.kubesphere.jenkins.devops.auth.KubesphereApiTokenAuthenticator.authenticate(KubesphereApiTokenAuthenticator.java:47)

 Caused: javax.servlet.ServletException

    at io.kubesphere.jenkins.devops.auth.KubesphereApiTokenAuthenticator.authenticate(KubesphereApiTokenAuthenticator.java:54)

    at jenkins.security.BasicHeaderAuthenticator.authenticate2(BasicHeaderAuthenticator.java:43)

    at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:83)

    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)

    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)

    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)

    at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:62)

    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)

    at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:109)

    at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)

    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

    at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:51)

    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

    at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:85)

    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

    at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)

    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

    at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:39)

    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)

    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)

    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578)

    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)

    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)

    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)

    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)

    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)

    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)

    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)

    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)

    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)

    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)

    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)

    at org.eclipse.jetty.server.Server.handle(Server.java:516)

    at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)

    at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)

    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)

    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)

    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)

    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)

    at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)

    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)

    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)

    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)

    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)

    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386)

    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)

    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)

    at java.base/java.lang.Thread.run(Thread.java:829)

Additional information

Kubernetes：v1.22.12
kubesphere: 3.4.1（member:3.3.1）
devops-apiserver:ks-v3.4.1
ks-jenkins:v3.4.0-2.319.3-1
other:dex:v2.29.0(Connected to Gilab and logged into Kubeshpere through Gilab account)
poet123 commented 6 months ago

Log in to Jenkins through the console and receive a username and password error message. The response log for devops Jenkins is as follows: 2024-04-08 01:34:23.748+0000 [id=3836] WARNING o.a.p.l.a.BindAuthenticator2#handleBindException: Failed to bind to LDAP: userDnuid=admin,ou=Users,dc=kubesphere,dc=io username=admin

javax.naming.AuthenticationException: [LDAP: error code 49 - Invalid Credentials]

at java.naming/com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3259)

at java.naming/com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3205)

at java.naming/com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2991)

at java.naming/com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2905)

at java.naming/com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:348)

at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxFromUrl(LdapCtxFactory.java:262)

at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:226)

at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:280)

at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:185)

at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:115)

at java.naming/javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:730)

at java.naming/javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:305)

at java.naming/javax.naming.InitialContext.init(InitialContext.java:236)

at java.naming/javax.naming.InitialContext.<init>(InitialContext.java:208)

at java.naming/javax.naming.directory.InitialDirContext.<init>(InitialDirContext.java:101)

at org.acegisecurity.ldap.DefaultInitialDirContextFactory.connect(DefaultInitialDirContextFactory.java:180)

at org.acegisecurity.ldap.DefaultInitialDirContextFactory.newInitialDirContext(DefaultInitialDirContextFactory.java:261)

at org.acegisecurity.ldap.LdapTemplate.execute(LdapTemplate.java:123)

at org.acegisecurity.ldap.LdapTemplate.retrieveEntry(LdapTemplate.java:165)

at org.acegisecurity.providers.ldap.authenticator.BindAuthenticator.bindWithDn(BindAuthenticator.java:87)

at org.acegisecurity.providers.ldap.authenticator.BindAuthenticator.authenticate(BindAuthenticator.java:72)

at org.acegisecurity.providers.ldap.authenticator.BindAuthenticator2.authenticate(BindAuthenticator2.java:49)

at org.acegisecurity.providers.ldap.LdapAuthenticationProvider.retrieveUser(LdapAuthenticationProvider.java:233)

at org.acegisecurity.providers.dao.AbstractUserDetailsAuthenticationProvider$1.retrieveUser(AbstractUserDetailsAuthenticationProvider.java:52)

at org.springframework.security.authentication.dao.AbstractUserDetailsAuthenticationProvider.authenticate(AbstractUserDetailsAuthenticationProvider.java:133)

at org.acegisecurity.providers.dao.AbstractUserDetailsAuthenticationProvider.authenticate(AbstractUserDetailsAuthenticationProvider.java:66)

at org.acegisecurity.providers.ProviderManager.doAuthentication(ProviderManager.java:200)

at org.acegisecurity.AbstractAuthenticationManager.authenticate(AbstractAuthenticationManager.java:47)

at hudson.security.LDAPSecurityRealm$LDAPAuthenticationManager.authenticate(LDAPSecurityRealm.java:1019)

at org.acegisecurity.AuthenticationManager.lambda$toSpring$1(AuthenticationManager.java:48)

at org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter.attemptAuthentication(UsernamePasswordAuthenticationFilter.java:85)

at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:222)

at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212)

at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)

at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:97)

at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)

at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)

at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)

at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:62)

at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)

at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:109)

at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)

at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:51)

at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:85)

at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)

at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:39)

at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)

at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)

at org.eclipse.jetty.server.Server.handle(Server.java:516)

at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)

at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)

at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)

at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)

at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)

at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)

at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386)

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)

at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)

at java.base/java.lang.Thread.run(Thread.java:829)
yudong2015 commented 3 months ago

Please check if there is user 'wukai1' in KubeSphere Console ？If exist, try to update password of user 'wukai1' ;