Open jvalkeal opened 4 years ago
eskuai
commented
4 years ago Changed, updated and tested. Skipper still fails Readme file updated with changes, description and messages logging from scdf and Skipper. TX @jvalkeal and @sabbyanandan . I keep testing
jvalkeal
commented
4 years ago Right, I just tried it out with a real instance and this is a very simple and probably slightly dummy configuration but it seems to work. https://github.com/jvalkeal/randomstuff/tree/master/dataflow-keycloak
It probably should give ideas of your environment if you at least get something working.
jvalkeal
commented
4 years ago You could check what you have in a client config as I think I saw 401's if I didn't have those Service Accounts Enabled and Authorization Enabled checked.
I've never seen Connection refused errors in a setup like this so not sure if it's k8s related. Also isn---t active is a new kind of error I've yet to see in my setups.
I'm curious what is a proper and production ready keycloak setup where dataflow should work. Keycloak seems to have so many ways to set things up and many bells and whistles to configure or enable/disable.
eskuai
commented
4 years ago 
Another test
using another realm , .. scdf23
2020-01-18 18:01:36.700 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/jobs/thinexecutions'
2020-01-18 18:01:36.700 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/jobs/instances'
2020-01-18 18:01:36.700 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/jobs/instances/*'
2020-01-18 18:01:36.700 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/runtime/streams'
2020-01-18 18:01:36.700 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/runtime/apps'
2020-01-18 18:01:36.700 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/runtime/apps/**'
2020-01-18 18:01:36.700 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/streams/definitions'
2020-01-18 18:01:36.701 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.a.i.FilterSecurityInterceptor : Secure object: FilterInvocation: URL: /streams/definitions?page=0&size=30&sort=name,ASC; Attributes: [hasRole('ROLE_VIEW')]
2020-01-18 18:01:36.701 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.a.i.FilterSecurityInterceptor : Previously Authenticated: org.springframework.security.oauth2.client.authentication.OAuth2AuthenticationToken@628cc956: Principal: Name: [mk], Granted Authorities: [[ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW]], User Attributes: [{sub=40e9e738-a11c-4508-ab32-e1860ea74782, email_verified=false, user_name=mk, iss=http://keycloak-http:8844/auth/realms/scdf23, typ=ID, preferred_username=mk, nonce=D4rbaKGtasXfDF5b2JBjo5Foy_PQo6liO3_4_5r59_0, aud=[dataflow], acr=1, nbf=Thu Jan 01 01:00:00 CET 1970, azp=dataflow, auth_time=2020-01-18T17:01:13Z, exp=2020-01-18T17:06:13Z, session_state=3f0cb184-ea66-4740-8f0a-c4cbea35de65, iat=2020-01-18T17:01:13Z, jti=851210c7-4297-4298-a929-d678d24b2c13}]; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@fffed504: RemoteIpAddress: 127.0.0.1; SessionId: 5E0D8B80FD764EC3293CC0040D7E061B; Granted Authorities: ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW
2020-01-18 18:01:36.701 DEBUG 1 --- [nio-8080-exec-2] o.s.s.access.vote.AffirmativeBased : Voter: org.springframework.security.web.access.expression.WebExpressionVoter@6a716730, returned: 1
2020-01-18 18:01:36.701 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.a.i.FilterSecurityInterceptor : Authorization successful
2020-01-18 18:01:36.701 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.a.i.FilterSecurityInterceptor : RunAsManager did not change Authentication object
2020-01-18 18:01:36.701 DEBUG 1 --- [nio-8080-exec-2] o.s.security.web.FilterChainProxy : /streams/definitions?page=0&size=30&sort=name,ASC reached end of additional filter chain; proceeding with original chain
2020-01-18 18:01:37.087 ERROR 1 --- [nio-8080-exec-2] o.s.c.d.s.c.RestControllerAdvice : Caught exception while handling a request
java.lang.IllegalArgumentException: dslText is required
at org.springframework.util.Assert.hasText(Assert.java:284)
at org.springframework.cloud.dataflow.core.StreamDefinition.<init>(StreamDefinition.java:98)
at org.springframework.cloud.dataflow.rest.util.ArgumentSanitizer.sanitizeOriginalStreamDsl(ArgumentSanitizer.java:166)
at org.springframework.cloud.dataflow.server.controller.StreamDefinitionController$Assembler.instantiateModel(StreamDefinitionController.java:210)
at org.springframework.cloud.dataflow.server.controller.StreamDefinitionController$Assembler.instantiateModel(StreamDefinitionController.java:185)
at org.springframework.hateoas.server.mvc.RepresentationModelAssemblerSupport.createModelWithId(RepresentationModelAssemblerSupport.java:87)
at org.springframework.hateoas.server.mvc.RepresentationModelAssemblerSupport.createModelWithId(RepresentationModelAssemblerSupport.java:79)
at org.springframework.cloud.dataflow.server.controller.StreamDefinitionController$Assembler.toModel(StreamDefinitionController.java:199)
at org.springframework.cloud.dataflow.server.controller.StreamDefinitionController$Assembler.toModel(StreamDefinitionController.java:185)
at org.springframework.data.web.PagedResourcesAssembler.createModel(PagedResourcesAssembler.java:211)
at org.springframework.data.web.PagedResourcesAssembler.toModel(PagedResourcesAssembler.java:123)
at org.springframework.cloud.dataflow.server.controller.StreamDefinitionController.list(StreamDefinitionController.java:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:888)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:103)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:154)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:154)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:114)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.ui.DefaultLogoutPageGeneratingFilter.doFilterInternal(DefaultLogoutPageGeneratingFilter.java:52)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:216)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.oauth2.client.web.OAuth2AuthorizationRequestRedirectFilter.doFilterInternal(OAuth2AuthorizationRequestRedirectFilter.java:160)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:108)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:526)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1579)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
2020-01-18 18:01:37.092 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.header.writers.HstsHeaderWriter : Not injecting HSTS header since it did not match the requestMatcher org.springframework.security.web.header.writers.HstsHeaderWriter$SecureRequestMatcher@674a5c74
2020-01-18 18:01:37.093 DEBUG 1 --- [nio-8080-exec-2] o.s.s.w.a.ExceptionTranslationFilter : Chain processed normally
2020-01-18 18:01:37.094 DEBUG 1 --- [nio-8080-exec-2] s.s.w.c.SecurityContextPersistenceFilter : SecurityContextHolder now cleared, as request processing completed
2020-01-18 18:01:42.767 DEBUG 1 --- [nio-8080-exec-8] o.s.security.web.FilterChainProxy : /management/health at position 1 of 17 in additional filter chain; firing Filter: 'WebAsyncManagerIntegrationFilter'
2020-01-18 18:01:42.767 DEBUG 1 --- [nio-8080-exec-8] o.s.security.web.FilterChainProxy : /management/health at position 2 of 17 in additional filter chain; firing Filter: 'SecurityContextPersistenceFilter'
2020-01-18 18:01:42.767 DEBUG 1 --- [nio-8080-exec-8] w.c.HttpSessionSecurityContextRepository : No HttpSession currently exists
2020-01-18 18:01:42.767 DEBUG 1 --- [nio-8080-exec-8] w.c.HttpSessionSecurityContextRepository : No SecurityContext was available from the HttpSession: null. A new one will be created.
2020-01-18 18:01:42.768 DEBUG 1 --- [nio-8080-exec-8] o.s.security.web.FilterChainProxy : /management/health at position 3 of 17 in additional filter chain; firing Filter: 'HeaderWriterFilter'
2020-01-18 18:01:42.768 DEBUG 1 --- [nio-8080-exec-8] o.s.security.web.FilterChainProxy : /management/health at position 4 of 17 in additional filter chain; firing Filter: 'LogoutFilter'
2020-01-18 18:01:42.768 DEBUG 1 --- [nio-8080-exec-8] o.s.s.web.util.matcher.OrRequestMatcher : Trying to match using Ant [pattern='/logout', GET]
2020-01-18 18:01:42.768 DEBUG 1 --- [nio-8080-exec-8] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/management/health'; against '/logout'
2020-01-18 18:01:42.768 DEBUG 1 --- [nio-8080-exec-8] o.s.s.web.util.matcher.OrRequestMatcher : Trying to match using Ant [pattern='/logout', POST]
2020-01-18 18:01:42.768 DEBUG 1 --- [nio-8080-exec-8] o.s.s.w.u.matcher.AntPathRequestMatcher : Request 'GET /management/health' doesn't match 'POST /logout'
2020-01-18 18:01:42.768 DEBUG 1 --- [nio-8080-exec-8] o.s.s.web.util.matcher.OrRequestMatcher : Trying to match using Ant [pattern='/logout', PUT]
2020-01-18 18:01:42.768 DEBUG 1 --- [nio-8080-exec-8] o.s.s.w.u.matcher.AntPathRequestMatcher : Request 'GET /management/health' doesn't match 'PUT /logout'
2020-01-18 18:01:42.768 DEBUG 1 --- [nio-8080-exec-8] o.s.s.web.util.matcher.OrRequestMatcher : Trying to match using Ant [pattern='/logout', DELETE]
2020-01-18 18:01:42.768 DEBUG 1 --- [nio-8080-exec-8] o.s.s.w.u.matcher.AntPathRequestMatcher : Request 'GET /management/health' doesn't match 'DELETE /logout'
2020-01-18 18:01:42.768 DEBUG 1 --- [nio-8080-exec-8] o.s.s.web.util.matcher.OrRequestMatcher : No matches found
sabbyanandan
commented
4 years ago Looking at Previously Authenticated: and Authorization successful logs, it appears both of those workflows are successful against your identity backend (keycloak).
What you see below as error is completely unrelated to security — at least from what I see.
Maybe you have existing stream definitions with incomplete data (eg: no DSL text — I do not know how this is even possible, however) in the database.
eskuai
commented
4 years ago I'll check again, but on k8s there is no stream deployed and I don t remember to deploy any stream (not working) Follow before problem, our friend "401"
eskuai
commented
4 years ago could be the keycloak version ? using 8.0.1 ... i am looking for a helm for 7.0.1 and I'll config DEBUG logging could be something about deployment vs config ... I am gooing to remove properties from deployement and all config in server-config.
sabbyanandan
commented
4 years ago @eskuai: To avoid going back and forth like how we did in Gitter, I'd recommend leaving everything behind. Just start with SCDF locally. Everything outside of K8s, and eliminate all the related complexities/confusion.
Use this new sample from @jvalkeal: https://github.com/jvalkeal/randomstuff/tree/master/dataflow-keycloak
As next steps, 1) Use the sample as-is (don't change or customize anything). See if you can get this working first. 2) If 1 works, switch the config values to "your" keycloak; it doesn't matter what version .. just swap the values in the same sample with "your" keycloak values.
If 1 and 2 works, you can then figure out how to make this work on K8s, in your environment.
Unless we do the most basic testing and prove that it works in such a simplistic setup, we will end up going in circles, and spending a lot of time.
Just my 2-cents.
eskuai
commented
4 years ago Ok, 1) Test locally , do you prefer locally jars or minikube? ... I'll apply yaml from jvalkeal . No changes keycloak from docker as jvalkeal shows
2) if ok, then only change url to my keycloak
Ok, let's check and test
Tx
sabbyanandan
commented
4 years ago I'd run exactly how it is defined in the README. Let's get that up and running your box. No K8s, and no minikube yet.
jvalkeal
commented
4 years ago I'd indeed try to get things working outside of k8s and when that is working then moving into k8s. This way there's less things to break things up.
jvalkeal
commented
4 years ago I was looking key-cfg-dataflow-4.png and you assigned via Realm Roles. Not really sure how it should work but I assigned via Client Roles which you see from its dropdown menu.
eskuai
commented
4 years ago Right, i had seen in your config too and i've checked both with same result. It is a way that, i think, you described from a issue about keycloak.
eskuai
commented
4 years ago Hi @sabbyanandan @jvalkeal
Use the sample as-is (don't change or customize anything). See if you can get this working first.
Tested following instructions from https://github.com/jvalkeal/randomstuff/tree/master/dataflow-keycloak and it works like a charm.
No problem, 0 ... including tested with 2.3.0 and 2.2.1
now, i am going to do step 2
eskuai
commented
4 years ago 2) If 1 works, switch the config values to "your" keycloak; it doesn't matter what version .. just swap the values in the same sample with "your" keycloak values.
Using jars and keycloak with my config on k8s ... It is WORKS... The only change i've made into my kc config is change "localhost:9393/*" and applying scdf23 realm into yaml ( and updated credentials). The keycloak is mk vs ADMIN
um... --debug is applied but i cannot read about debug security logging...
spring:
cloud:
skipper:
security:
authorization:
provider-role-mappings:
keycloak:
map-oauth-scopes: true
role-mappings:
ROLE_VIEW: dataflow.view
ROLE_CREATE: dataflow.create
ROLE_MANAGE: dataflow.manage
ROLE_DEPLOY: dataflow.deploy
ROLE_DESTROY: dataflow.destroy
ROLE_MODIFY: dataflow.modify
ROLE_SCHEDULE: dataflow.schedule
security:
oauth2:
client:
registration:
keycloak:
redirect-uri: '{baseUrl}/login/oauth2/code/{registrationId}'
authorization-grant-type: authorization_code
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
scope:
- openid
provider:
keycloak:
jwk-set-uri: http://localhost:8844/auth/realms/scdf23/protocol/openid-connect/certs
token-uri: http://localhost:8844/auth/realms/scdf23/protocol/openid-connect/token
user-info-uri: http://localhost:8844/auth/realms/scdf23/protocol/openid-connect/userinfo
user-name-attribute: user_name
authorization-uri: http://localhost:8844/auth/realms/scdf23/protocol/openid-connect/auth
resourceserver:
opaquetoken:
introspection-uri: http://localhost:8844/auth/realms/scdf23/protocol/openid-connect/token/introspect
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
and
spring:
cloud:
dataflow:
security:
authorization:
provider-role-mappings:
keycloak:
map-oauth-scopes: true
role-mappings:
ROLE_VIEW: dataflow.view
ROLE_CREATE: dataflow.create
ROLE_MANAGE: dataflow.manage
ROLE_DEPLOY: dataflow.deploy
ROLE_DESTROY: dataflow.destroy
ROLE_MODIFY: dataflow.modify
ROLE_SCHEDULE: dataflow.schedule
security:
oauth2:
client:
registration:
keycloak:
redirect-uri: '{baseUrl}/login/oauth2/code/{registrationId}'
authorization-grant-type: authorization_code
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
scope:
- openid
- dataflow.view
- dataflow.deploy
- dataflow.destroy
- dataflow.manage
- dataflow.modify
- dataflow.schedule
- dataflow.create
provider:
keycloak:
jwk-set-uri: http://localhost:8844/auth/realms/scdf23/protocol/openid-connect/certs
token-uri: http://localhost:8844/auth/realms/scdf23/protocol/openid-connect/token
user-info-uri: http://localhost:8844/auth/realms/scdf23/protocol/openid-connect/userinfo
user-name-attribute: user_name
authorization-uri: http://localhost:8844/auth/realms/scdf23/protocol/openid-connect/auth
resourceserver:
opaquetoken:
introspection-uri: http://localhost:8844/auth/realms/scdf23/protocol/openid-connect/token/introspect
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
authorization:
check-token-access: isAuthenticated()Then
Next step, number 3, could be update all skipper and scdf2 deployement and config , removing all properties from deployements files and settingit it into config again?
eskuai
commented
4 years ago 
2020-01-19 21:14:44.084 DEBUG 1 --- [nio-8080-exec-3] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/dashboard/favicon.ico'; against '/dashboard'
2020-01-19 21:14:44.084 DEBUG 1 --- [nio-8080-exec-3] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/dashboard/favicon.ico'; against '/dashboard/**'
2020-01-19 21:14:44.084 DEBUG 1 --- [nio-8080-exec-3] o.s.s.w.a.i.FilterSecurityInterceptor : Secure object: FilterInvocation: URL: /dashboard/favicon.ico; Attributes: [authenticated]
2020-01-19 21:14:44.084 DEBUG 1 --- [nio-8080-exec-3] o.s.s.w.a.i.FilterSecurityInterceptor : Previously Authenticated: org.springframework.security.oauth2.client.authentication.OAuth2AuthenticationToken@12dd4050: Principal: Name: [mk], Granted Authorities: [[ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW]], User Attributes: [{sub=40e9e738-a11c-4508-ab32-e1860ea74782, email_verified=false, user_name=mk, iss=http://keycloak-http:8844/auth/realms/scdf23, typ=ID, preferred_username=mk, nonce=G7ynbiWMri33thNbDJ-PWaHZY1yjTQITS_pZnquHegE, aud=[dataflow], acr=1, nbf=Thu Jan 01 01:00:00 CET 1970, azp=dataflow, auth_time=2020-01-19T20:11:48Z, exp=2020-01-19T20:16:48Z, session_state=e9e39608-120d-429d-bff6-996ca6001434, iat=2020-01-19T20:11:48Z, jti=487749b8-569c-4d05-aca5-c64e66a0f4ea}]; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@0: RemoteIpAddress: 127.0.0.1; SessionId: 9ADEBB0C1D00A3FC93A870A61DCE920A; Granted Authorities: ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW
2020-01-19 21:14:44.084 DEBUG 1 --- [nio-8080-exec-3] o.s.s.access.vote.AffirmativeBased : Voter: org.springframework.security.web.access.expression.WebExpressionVoter@2cf6925f, returned: 1
2020-01-19 21:14:44.084 DEBUG 1 --- [nio-8080-exec-3] o.s.s.w.a.i.FilterSecurityInterceptor : Authorization successful
2020-01-19 21:14:44.084 DEBUG 1 --- [nio-8080-exec-3] o.s.s.w.a.i.FilterSecurityInterceptor : RunAsManager did not change Authentication object
2020-01-19 21:14:44.084 DEBUG 1 --- [nio-8080-exec-3] o.s.security.web.FilterChainProxy : /dashboard/favicon.ico reached end of additional filter chain; proceeding with original chain
2020-01-19 21:14:44.083 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/jobs/thinexecutions'
2020-01-19 21:14:44.085 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/jobs/instances'
2020-01-19 21:14:44.086 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/jobs/instances/*'
2020-01-19 21:14:44.086 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/runtime/streams'
2020-01-19 21:14:44.086 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/runtime/apps'
2020-01-19 21:14:44.086 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/runtime/apps/**'
2020-01-19 21:14:44.086 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/streams/definitions'
2020-01-19 21:14:44.086 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.a.i.FilterSecurityInterceptor : Secure object: FilterInvocation: URL: /streams/definitions?page=0&size=30&sort=name,ASC; Attributes: [hasRole('ROLE_VIEW')]
2020-01-19 21:14:44.086 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.a.i.FilterSecurityInterceptor : Previously Authenticated: org.springframework.security.oauth2.client.authentication.OAuth2AuthenticationToken@12dd4050: Principal: Name: [mk], Granted Authorities: [[ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW]], User Attributes: [{sub=40e9e738-a11c-4508-ab32-e1860ea74782, email_verified=false, user_name=mk, iss=http://keycloak-http:8844/auth/realms/scdf23, typ=ID, preferred_username=mk, nonce=G7ynbiWMri33thNbDJ-PWaHZY1yjTQITS_pZnquHegE, aud=[dataflow], acr=1, nbf=Thu Jan 01 01:00:00 CET 1970, azp=dataflow, auth_time=2020-01-19T20:11:48Z, exp=2020-01-19T20:16:48Z, session_state=e9e39608-120d-429d-bff6-996ca6001434, iat=2020-01-19T20:11:48Z, jti=487749b8-569c-4d05-aca5-c64e66a0f4ea}]; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@0: RemoteIpAddress: 127.0.0.1; SessionId: 9ADEBB0C1D00A3FC93A870A61DCE920A; Granted Authorities: ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW
2020-01-19 21:14:44.086 DEBUG 1 --- [nio-8080-exec-5] o.s.s.access.vote.AffirmativeBased : Voter: org.springframework.security.web.access.expression.WebExpressionVoter@2cf6925f, returned: 1
2020-01-19 21:14:44.086 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.a.i.FilterSecurityInterceptor : Authorization successful
2020-01-19 21:14:44.087 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.a.i.FilterSecurityInterceptor : RunAsManager did not change Authentication object
2020-01-19 21:14:44.087 DEBUG 1 --- [nio-8080-exec-5] o.s.security.web.FilterChainProxy : /streams/definitions?page=0&size=30&sort=name,ASC reached end of additional filter chain; proceeding with original chain
2020-01-19 21:14:44.092 DEBUG 1 --- [nio-8080-exec-3] o.s.s.w.header.writers.HstsHeaderWriter : Not injecting HSTS header since it did not match the requestMatcher org.springframework.security.web.header.writers.HstsHeaderWriter$SecureRequestMatcher@3a731600
2020-01-19 21:14:44.092 DEBUG 1 --- [nio-8080-exec-9] o.s.s.w.header.writers.HstsHeaderWriter : Not injecting HSTS header since it did not match the requestMatcher org.springframework.security.web.header.writers.HstsHeaderWriter$SecureRequestMatcher@3a731600
2020-01-19 21:14:44.093 DEBUG 1 --- [nio-8080-exec-9] o.s.s.w.a.ExceptionTranslationFilter : Chain processed normally
2020-01-19 21:14:44.094 DEBUG 1 --- [nio-8080-exec-9] s.s.w.c.SecurityContextPersistenceFilter : SecurityContextHolder now cleared, as request processing completed
2020-01-19 21:14:44.098 DEBUG 1 --- [nio-8080-exec-3] o.s.s.w.a.ExceptionTranslationFilter : Chain processed normally
2020-01-19 21:14:44.098 DEBUG 1 --- [nio-8080-exec-3] s.s.w.c.SecurityContextPersistenceFilter : SecurityContextHolder now cleared, as request processing completed
2020-01-19 21:14:44.224 ERROR 1 --- [nio-8080-exec-5] o.s.c.d.s.c.RestControllerAdvice : Caught exception while handling a request
org.springframework.cloud.skipper.SkipperException: No deployer named 'default'
at org.springframework.cloud.skipper.client.SkipperClientResponseErrorHandler.handleSkipperException(SkipperClientResponseErrorHandler.java:109)
at org.springframework.cloud.skipper.client.SkipperClientResponseErrorHandler.handleError(SkipperClientResponseErrorHandler.java:76)
at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:785)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:743)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:698)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:625)
at org.springframework.cloud.skipper.client.DefaultSkipperClient.status(DefaultSkipperClient.java:133)
at org.springframework.cloud.dataflow.server.stream.SkipperStreamDeployer.getStreamDeploymentState(SkipperStreamDeployer.java:170)
at org.springframework.cloud.dataflow.server.stream.SkipperStreamDeployer.streamsStates(SkipperStreamDeployer.java:159)
at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService.state(DefaultStreamService.java:332)
at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService$$FastClassBySpringCGLIB$$89697014.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService$$EnhancerBySpringCGLIB$$654ef539.state(<generated>)
at org.springframework.cloud.dataflow.server.controller.StreamDefinitionController$Assembler.<init>(StreamDefinitionController.java:192)
at org.springframework.cloud.dataflow.server.controller.StreamDefinitionController.list(StreamDefinitionController.java:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:888)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:103)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:154)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:154)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:114)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.ui.DefaultLogoutPageGeneratingFilter.doFilterInternal(DefaultLogoutPageGeneratingFilter.java:52)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:216)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.oauth2.client.web.OAuth2AuthorizationRequestRedirectFilter.doFilterInternal(OAuth2AuthorizationRequestRedirectFilter.java:160)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:108)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:526)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1579)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
2020-01-19 21:14:44.226 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.header.writers.HstsHeaderWriter : Not injecting HSTS header since it did not match the requestMatcher org.springframework.security.web.header.writers.HstsHeaderWriter$SecureRequestMatcher@3a731600
2020-01-19 21:14:44.226 DEBUG 1 --- [nio-8080-exec-5] o.s.s.w.a.ExceptionTranslationFilter : Chain processed normally
2020-01-19 21:14:44.227 DEBUG 1 --- [nio-8080-exec-5] s.s.w.c.SecurityContextPersistenceFilter : SecurityContextHolder now cleared, as request processing completed
2020-01-19 21:14:44.264 DEBUG 1 --- [io-8080-exec-10] o.s.s.w.header.writers.HstsHeaderWriter : Not injecting HSTS header since it did not match the requestMatcher org.springframework.security.web.header.writers.HstsHeaderWriter$SecureRequestMatcher@3a731600
2020-01-19 21:14:44.265 DEBUG 1 --- [io-8080-exec-10] o.s.s.w.a.ExceptionTranslationFilter : Chain processed normally
2020-01-19 21:14:44.265 DEBUG 1 --- [io-8080-exec-10] s.s.w.c.SecurityContextPersistenceFilter : SecurityContextHolder now cleared, as request processing completed
2020-01-19 21:14:44.360 DEBUG 1 --- [nio-8080-exec-1] o.s.security.web.FilterChainProxy : /dashboard/fontawesome-webfont.e6cf7c6ec7c2d6f670ae.woff2?v=4.6.3 at position 1 of 17 in additional filter chain; firing Filter: 'WebAsyncManagerIntegrationFilter'
2020-01-19 21:14:44.360 DEBUG 1 --- [nio-8080-exec-1] o.s.security.web.FilterChainProxy : /dashboard/fontawesome-webfont.e6cf7c6ec7c2d6f670ae.woff2?v=4.6.3 at position 2 of 17 in additional filter chain; firing Filter: 'SecurityContextPersistenceFilter'
2020-01-19 21:14:44.361 DEBUG 1 --- [nio-8080-exec-1] w.c.HttpSessionSecurityContextRepository : Obtained a valid SecurityContext from SPRING_SECURITY_CONTEXT: 'org.springframework.security.core.context.SecurityContextImpl@12dd4050: Authentication: org.springframework.security.oauth2.client.authentication.OAuth2AuthenticationToken@12dd4050: Principal: Name: [mk], Granted Authorities: [[ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW]], User Attributes: [{sub=40e9e738-a11c-4508-ab32-e1860ea74782, email_verified=false, user_name=mk, iss=http://keycloak-http:8844/auth/realms/scdf23, typ=ID, preferred_username=mk, nonce=G7ynbiWMri33thNbDJ-PWaHZY1yjTQITS_pZnquHegE, aud=[dataflow], acr=1, nbf=Thu Jan 01 01:00:00 CET 1970, azp=dataflow, auth_time=2020-01-19T20:11:48Z, exp=2020-01-19T20:16:48Z, session_state=e9e39608-120d-429d-bff6-996ca6001434, iat=2020-01-19T20:11:48Z, jti=487749b8-569c-4d05-aca5-c64e66a0f4ea}]; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@0: RemoteIpAddress: 127.0.0.1; SessionId: 9ADEBB0C1D00A3FC93A870A61DCE920A; Granted Authorities: ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW'
2020-01-19 21:14:44.361 DEBUG 1 --- [nio-8080-exec-1] o.s.security.web.FilterChainProxy : /dashboard/fontawesome-webfont.e6cf7c6ec7c2d6f670ae.woff2?v=4.6.3 at position 3 of 17 in additional filter chain; firing Filter: 'HeaderWriterFilter'
2020-01-19 21:14:44.361 DEBUG 1 --- [nio-8080-exec-1] o.s.security.web.FilterChainProxy : /dashboard/fontawesome-webfont.e6cf7c6ec7c2d6f670ae.woff2?v=4.6.3 at position 4 of 17 in additional filter chain; firing Filter: 'LogoutFilter'
and
2020-01-19 21:12:20.727 DEBUG 1 --- [nio-8080-exec-9] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/streams/definitions'
2020-01-19 21:12:20.727 DEBUG 1 --- [nio-8080-exec-9] o.s.s.w.a.i.FilterSecurityInterceptor : Secure object: FilterInvocation: URL: /streams/definitions?page=0&size=30&sort=name,ASC; Attributes: [hasRole('ROLE_VIEW')]
2020-01-19 21:12:20.727 DEBUG 1 --- [nio-8080-exec-9] o.s.s.w.a.i.FilterSecurityInterceptor : Previously Authenticated: org.springframework.security.oauth2.client.authentication.OAuth2AuthenticationToken@12dd4050: Principal: Name: [mk], Granted Authorities: [[ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW]], User Attributes: [{sub=40e9e738-a11c-4508-ab32-e1860ea74782, email_verified=false, user_name=mk, iss=http://keycloak-http:8844/auth/realms/scdf23, typ=ID, preferred_username=mk, nonce=G7ynbiWMri33thNbDJ-PWaHZY1yjTQITS_pZnquHegE, aud=[dataflow], acr=1, nbf=Thu Jan 01 01:00:00 CET 1970, azp=dataflow, auth_time=2020-01-19T20:11:48Z, exp=2020-01-19T20:16:48Z, session_state=e9e39608-120d-429d-bff6-996ca6001434, iat=2020-01-19T20:11:48Z, jti=487749b8-569c-4d05-aca5-c64e66a0f4ea}]; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@0: RemoteIpAddress: 127.0.0.1; SessionId: 9ADEBB0C1D00A3FC93A870A61DCE920A; Granted Authorities: ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW
2020-01-19 21:12:20.727 DEBUG 1 --- [nio-8080-exec-9] o.s.s.access.vote.AffirmativeBased : Voter: org.springframework.security.web.access.expression.WebExpressionVoter@2cf6925f, returned: 1
2020-01-19 21:12:20.727 DEBUG 1 --- [nio-8080-exec-9] o.s.s.w.a.i.FilterSecurityInterceptor : Authorization successful
2020-01-19 21:12:20.727 DEBUG 1 --- [nio-8080-exec-9] o.s.s.w.a.i.FilterSecurityInterceptor : RunAsManager did not change Authentication object
2020-01-19 21:12:20.727 DEBUG 1 --- [nio-8080-exec-9] o.s.security.web.FilterChainProxy : /streams/definitions?page=0&size=30&sort=name,ASC reached end of additional filter chain; proceeding with original chain
2020-01-19 21:12:21.742 ERROR 1 --- [nio-8080-exec-9] o.s.c.d.s.c.RestControllerAdvice : Caught exception while handling a request
org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://10.107.158.21/api/release/status/tl": Connect to 10.107.158.21:80 [/10.107.158.21] failed: Connection refused (Connection refused); nested exception is org.apache.http.conn.HttpHostConnectException: Connect to 10.107.158.21:80 [/10.107.158.21] failed: Connection refused (Connection refused)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:751)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:698)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:625)
at org.springframework.cloud.skipper.client.DefaultSkipperClient.status(DefaultSkipperClient.java:133)
at org.springframework.cloud.dataflow.server.stream.SkipperStreamDeployer.getStreamDeploymentState(SkipperStreamDeployer.java:170)
at org.springframework.cloud.dataflow.server.stream.SkipperStreamDeployer.streamsStates(SkipperStreamDeployer.java:159)
at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService.state(DefaultStreamService.java:332)
at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService$$FastClassBySpringCGLIB$$89697014.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService$$EnhancerBySpringCGLIB$$654ef539.state(<generated>)
at org.springframework.cloud.dataflow.server.controller.StreamDefinitionController$Assembler.<init>(StreamDefinitionController.java:192)
at org.springframework.cloud.dataflow.server.controller.StreamDefinitionController.list(StreamDefinitionController.java:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:888)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:103)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:154)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:154)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:114)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.ui.DefaultLogoutPageGeneratingFilter.doFilterInternal(DefaultLogoutPageGeneratingFilter.java:52)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:216)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.oauth2.client.web.OAuth2AuthorizationRequestRedirectFilter.doFilterInternal(OAuth2AuthorizationRequestRedirectFilter.java:160)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:108)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:526)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1579)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 10.107.158.21:80 [/10.107.158.21] failed: Connection refused (Connection refused)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:374)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:87)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
at org.springframework.http.client.InterceptingClientHttpRequest$InterceptingRequestExecution.execute(InterceptingClientHttpRequest.java:109)
at org.springframework.cloud.common.security.core.support.OAuth2AccessTokenProvidingClientHttpRequestInterceptor.intercept(OAuth2AccessTokenProvidingClientHttpRequestInterceptor.java:72)
at org.springframework.http.client.InterceptingClientHttpRequest$InterceptingRequestExecution.execute(InterceptingClientHttpRequest.java:93)
at org.springframework.boot.actuate.metrics.web.client.MetricsClientHttpRequestInterceptor.intercept(MetricsClientHttpRequestInterceptor.java:93)
at org.springframework.http.client.InterceptingClientHttpRequest$InterceptingRequestExecution.execute(InterceptingClientHttpRequest.java:93)
at org.springframework.http.client.InterceptingClientHttpRequest.executeInternal(InterceptingClientHttpRequest.java:77)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:742)
... 127 common frames omitted
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
... 148 common frames omitted
2020-01-19 21:12:21.744 DEBUG 1 --- [nio-8080-exec-9] o.s.s.w.header.writers.HstsHeaderWriter : Not injecting HSTS header since it did not match the requestMatcher org.springframework.security.web.header.writers.HstsHeaderWriter$SecureRequestMatcher@3a731600
2020-01-19 21:12:21.744 DEBUG 1 --- [nio-8080-exec-9] o.s.s.w.a.ExceptionTranslationFilter : Chain processed normally
2020-01-19 21:12:21.744 DEBUG 1 --- [nio-8080-exec-9] s.s.w.c.SecurityContextPersistenceFilter : SecurityContextHolder now cleared, as request processing completed
And files
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "scdf.fullname" . }}-server
labels:
app: {{ template "scdf.name" . }}
component: server
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
data:
application.yaml: |-
management:
endpoints:
web:
base-path: /management
security:
roles: MANAGE
spring:
cloud:
dataflow:
applicationProperties:
stream:
management:
{{- if .Values.prometheus.enabled }}
metrics:
export:
prometheus:
enabled: true
{{- end }}
endpoints:
web:
exposure:
include: 'prometheus,info,health'
spring:
cloud:
streamapp:
security:
enabled: false
{{- if .Values.grafana.enabled }}
grafana-info:
url: '{{ .Values.grafana.url }}'
{{- end }}
task:
platform:
kubernetes:
accounts:
{{ .Values.server.platformName }}:
environmentVariables:
- JAVA_TOOL_OPTIONS=-Xmx1024m -Xms1024m -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Duser.timezone=Europe/Madrid -XX:-TieredCompilation -XX:TieredStopAtLevel=1 -XX:+UseCompressedOops -XX:+UseCompressedClassPointers -Xverify:none -XX:+AggressiveOpts -XX:+UseG1GC -XX:+UseStringDeduplication
- TZ=Europe/Madrid
maximum-concurrent-tasks: 200
lifecycle:
preStop:
exec:
command: ["/bin/sh","-c","sleep 10"]
limits:
memory: {{ .Values.deployer.resourceLimits.memory }}
cpu: {{ .Values.deployer.resourceLimits.cpu }}
security:
authorization:
provider-role-mappings:
keycloak:
map-oauth-scopes: true
role-mappings:
ROLE_VIEW: dataflow.view
ROLE_CREATE: dataflow.create
ROLE_MANAGE: dataflow.manage
ROLE_DEPLOY: dataflow.deploy
ROLE_DESTROY: dataflow.destroy
ROLE_MODIFY: dataflow.modify
ROLE_SCHEDULE: dataflow.schedule
security:
oauth2:
client:
registration:
keycloak:
redirect-uri: '{baseUrl}/login/oauth2/code/{registrationId}'
authorization-grant-type: authorization_code
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
scope:
- openid
- dataflow.view
- dataflow.deploy
- dataflow.destroy
- dataflow.manage
- dataflow.modify
- dataflow.schedule
- dataflow.create
provider:
keycloak:
jwk-set-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/certs
token-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/token
user-info-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/userinfo
user-name-attribute: user_name
authorization-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/auth
resourceserver:
opaquetoken:
introspection-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/token/introspect
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
authorization:
check-token-access: isAuthenticated()
datasource:
url: 'jdbc:{{ template "scdf.database.scheme" . }}://{{ template "scdf.database.host" . }}:{{ template "scdf.database.port" . }}/{{ template "scdf.database.dataflow" . }}'
driverClassName: {{ template "scdf.database.driver" . }}
username: {{ template "scdf.database.user" . }}
password: {{ template "scdf.database.password" . }}
hikari.maximumPoolSize: 60
testOnBorrow: true
validationQuery: "SELECT 1"
and
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "scdf.fullname" . }}-skipper
labels:
app: {{ template "scdf.name" . }}
component: skipper
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
data:
application.yaml: |-
spring:
cloud:
skipper:
security:
authorization:
permit-all-paths: "/actuator/**,/management/**,/authenticate,/security/info,/assets/**,/dashboard/logout-success-oauth.html"
provider-role-mappings:
keycloak:
map-oauth-scopes: true
role-mappings:
ROLE_VIEW: dataflow.view
ROLE_CREATE: dataflow.create
ROLE_MANAGE: dataflow.manage
ROLE_DEPLOY: dataflow.deploy
ROLE_DESTROY: dataflow.destroy
ROLE_MODIFY: dataflow.modify
ROLE_SCHEDULE: dataflow.schedule
security:
oauth2:
client:
registration:
keycloak:
redirect-uri: '{baseUrl}/login/oauth2/code/{registrationId}'
authorization-grant-type: authorization_code
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
scope:
- openid
- dataflow.view
- dataflow.deploy
- dataflow.destroy
- dataflow.manage
- dataflow.modify
- dataflow.schedule
- dataflow.create
provider:
keycloak:
jwk-set-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/certs
token-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/token
user-info-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/userinfo
user-name-attribute: user_name
authorization-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/auth
resourceserver:
opaquetoken:
introspection-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/token/introspect
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
server:
platform:
kubernetes:
accounts:
{{- $root := . -}}
{{- range .Values.skipper.platformName }}
{{ . }}:
namespace: {{ . }}
{{- $bucle := . -}}
{{- if eq $bucle "default" }}
deploymentServiceAccountName: {{ $root.Values.deployer.serviceAccountName }}
{{- else }}
deploymentServiceAccountName: {{ $root.Values.deployer.serviceAccountName }}-{{ . }}
{{- end }}
limits:
memory: {{ $root.Values.deployer.resourceLimits.memory }}
cpu: {{ $root.Values.deployer.resourceLimits.cpu }}
readinessProbePath: {{ $root.Values.deployer.readinessProbe.path }}
readinessProbeDelay: {{ $root.Values.deployer.readinessProbe.initialDelaySeconds }}
livenessProbePath: {{ $root.Values.deployer.livenessProbe.path }}
livenessProbeDelay: {{ $root.Values.deployer.livenessProbe.initialDelaySeconds }}
livenessProbePeriod: {{ $root.Values.deployer.livenessProbe.period }}
podAnnotations: 'prometheus.io/path:/actuator/prometheus,prometheus.io/port:8080,prometheus.io/scrape:true'
environmentVariables:
{{- if $root.Values.kafka.enabled }}
- SPRING_CLOUD_STREAM_KAFKA_BINDER_BROKERS={{ $root.Values.kafka.host }}:{{ $root.Values.kafka.port }}
- SPRING_CLOUD_STREAM_KAFKA_BINDER_ZK_NODES={{ $root.Values.kafka.zookeeper.host }}:{{ $root.Values.kafka.zookeeper.port }}
{{- end }}
- JAVA_TOOL_OPTIONS={{ $root.Values.deployer.javatoolopts }}
- TZ={{ $root.Values.deployer.tz }}
- SPRING_CLOUD_CONFIG_ENABLED=false
{{- end }}
applicationProperties:
stream:
management:
endpoints:
web:
exposure:
include: 'prometheus,info,health,management'
datasource:
url: 'jdbc:{{ template "scdf.database.scheme" . }}://{{ template "scdf.database.host" . }}:{{ template "scdf.database.port" . }}/{{ template "scdf.database.skipper" . }}'
driverClassName: {{ template "scdf.database.driver" . }}
username: {{ template "scdf.database.user" . }}
password: {{ template "scdf.database.password" . }}
testOnBorrow: true
hikari.maximumPoolSize: 30
validationQuery: "SELECT 1"
mon 20-01 , 12:03 GMT+2
Hi @jvalkeal @sabbyanandan ,
Checking yaml configs with another people from team, he thinks that configs yamls are not "valid". He thinks that yaml is sytax ok, but info about security or task is not right processed.
Updated again server-config and skipper-config,
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "scdf.fullname" . }}-skipper
labels:
app: {{ template "scdf.name" . }}
component: skipper
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
data:
application.yaml: |-
spring:
cloud:
skipper:
security:
authorization:
permit-all-paths: "/actuator/**,/management/**,/authenticate,/security/info,/assets/**,/dashboard/logout-success-oauth.html"
provider-role-mappings:
keycloak:
map-oauth-scopes: true
role-mappings:
ROLE_VIEW: dataflow.view
ROLE_CREATE: dataflow.create
ROLE_MANAGE: dataflow.manage
ROLE_DEPLOY: dataflow.deploy
ROLE_DESTROY: dataflow.destroy
ROLE_MODIFY: dataflow.modify
ROLE_SCHEDULE: dataflow.schedule
server:
platform:
kubernetes:
accounts:
{{- $root := . -}}
{{- range .Values.skipper.platformName }}
{{ . }}:
namespace: {{ . }}
{{- $bucle := . -}}
{{- if eq $bucle "default" }}
deploymentServiceAccountName: {{ $root.Values.deployer.serviceAccountName }}
{{- else }}
deploymentServiceAccountName: {{ $root.Values.deployer.serviceAccountName }}-{{ . }}
{{- end }}
limits:
memory: {{ $root.Values.deployer.resourceLimits.memory }}
cpu: {{ $root.Values.deployer.resourceLimits.cpu }}
readinessProbePath: {{ $root.Values.deployer.readinessProbe.path }}
readinessProbeDelay: {{ $root.Values.deployer.readinessProbe.initialDelaySeconds }}
livenessProbePath: {{ $root.Values.deployer.livenessProbe.path }}
livenessProbeDelay: {{ $root.Values.deployer.livenessProbe.initialDelaySeconds }}
livenessProbePeriod: {{ $root.Values.deployer.livenessProbe.period }}
podAnnotations: 'prometheus.io/path:/actuator/prometheus,prometheus.io/port:8080,prometheus.io/scrape:true'
environmentVariables:
{{- if $root.Values.kafka.enabled }}
- SPRING_CLOUD_STREAM_KAFKA_BINDER_BROKERS={{ $root.Values.kafka.host }}:{{ $root.Values.kafka.port }}
- SPRING_CLOUD_STREAM_KAFKA_BINDER_ZK_NODES={{ $root.Values.kafka.zookeeper.host }}:{{ $root.Values.kafka.zookeeper.port }}
{{- end }}
- JAVA_TOOL_OPTIONS={{ $root.Values.deployer.javatoolopts }}
- TZ={{ $root.Values.deployer.tz }}
- SPRING_CLOUD_CONFIG_ENABLED=false
{{- end }}
applicationProperties:
stream:
management:
endpoints:
web:
exposure:
include: 'prometheus,info,health,management'
security:
oauth2:
client:
registration:
keycloak:
redirect-uri: '{baseUrl}/login/oauth2/code/{registrationId}'
authorization-grant-type: authorization_code
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
scope:
- openid
- dataflow.view
- dataflow.deploy
- dataflow.destroy
- dataflow.manage
- dataflow.modify
- dataflow.schedule
- dataflow.create
provider:
keycloak:
jwk-set-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/certs
token-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/token
user-info-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/userinfo
user-name-attribute: user_name
authorization-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/auth
resourceserver:
opaquetoken:
introspection-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/token/introspect
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
jpa:
properties:
hibernate:
dialect: org.hibernate.dialect.MariaDB102Dialect
datasource:
url: 'jdbc:{{ template "scdf.database.scheme" . }}://{{ template "scdf.database.host" . }}:{{ template "scdf.database.port" . }}/{{ template "scdf.database.skipper" . }}'
driverClassName: {{ template "scdf.database.driver" . }}
username: {{ template "scdf.database.user" . }}
password: {{ template "scdf.database.password" . }}
testOnBorrow: true
hikari.maximumPoolSize: 30
validationQuery: "SELECT 1"and
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "scdf.fullname" . }}-server
labels:
app: {{ template "scdf.name" . }}
component: server
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
data:
application.yaml: |-
management:
endpoints:
web:
base-path: /management
security:
roles: MANAGE
spring:
cloud:
dataflow:
applicationProperties:
stream:
management:
{{- if .Values.prometheus.enabled }}
metrics:
export:
prometheus:
enabled: true
{{- end }}
endpoints:
web:
exposure:
include: 'prometheus,info,health'
spring:
cloud:
streamapp:
security:
enabled: false
{{- if .Values.grafana.enabled }}
grafana-info:
url: '{{ .Values.grafana.url }}'
{{- end }}
task:
platform:
kubernetes:
accounts:
{{ .Values.server.platformName }}:
environmentVariables:
- JAVA_TOOL_OPTIONS=-Xmx1024m -Xms1024m -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Duser.timezone=Europe/Madrid -XX:-TieredCompilation -XX:TieredStopAtLevel=1 -XX:+UseCompressedOops -XX:+UseCompressedClassPointers -Xverify:none -XX:+AggressiveOpts -XX:+UseG1GC -XX:+UseStringDeduplication
- TZ=Europe/Madrid
maximum-concurrent-tasks: 200
lifecycle:
preStop:
exec:
command: ["/bin/sh","-c","sleep 10"]
limits:
memory: {{ .Values.deployer.resourceLimits.memory }}
cpu: {{ .Values.deployer.resourceLimits.cpu }}
security:
authorization:
provider-role-mappings:
keycloak:
map-oauth-scopes: true
role-mappings:
ROLE_VIEW: dataflow.view
ROLE_CREATE: dataflow.create
ROLE_MANAGE: dataflow.manage
ROLE_DEPLOY: dataflow.deploy
ROLE_DESTROY: dataflow.destroy
ROLE_MODIFY: dataflow.modify
ROLE_SCHEDULE: dataflow.schedule
security:
oauth2:
client:
registration:
keycloak:
redirect-uri: '{baseUrl}/login/oauth2/code/{registrationId}'
authorization-grant-type: authorization_code
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
scope:
- openid
- dataflow.view
- dataflow.deploy
- dataflow.destroy
- dataflow.manage
- dataflow.modify
- dataflow.schedule
- dataflow.create
provider:
keycloak:
jwk-set-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/certs
token-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/token
user-info-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/userinfo
user-name-attribute: user_name
authorization-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/auth
resourceserver:
opaquetoken:
introspection-uri: http://keycloak-http:8844/auth/realms/scdf23/protocol/openid-connect/token/introspect
client-id: dataflow
client-secret: 2577e86a-cb98-46db-b0ff-d509dfa7e99f
authorization:
check-token-access: isAuthenticated()
jpa:
properties:
hibernate:
dialect: org.hibernate.dialect.MariaDB102Dialect
datasource:
url: 'jdbc:{{ template "scdf.database.scheme" . }}://{{ template "scdf.database.host" . }}:{{ template "scdf.database.port" . }}/{{ template "scdf.database.dataflow" . }}'
driverClassName: {{ template "scdf.database.driver" . }}
username: {{ template "scdf.database.user" . }}
password: {{ template "scdf.database.password" . }}
hikari.maximumPoolSize: 60
testOnBorrow: true
validationQuery: "SELECT 1"Another test, create dataflow database from scatch ,,. Then
1) There is a error if i use the name stream from past... local maven xxxx
2) Create stream can be use "one time"... I 've create a stream, and deplyed but next time to go to dashboard is freezed by our friend "401" http error code
2020-01-20 11:56:09.040 DEBUG 1 --- [io-8080-exec-10] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/jobs/instances/*'
2020-01-20 11:56:09.040 DEBUG 1 --- [io-8080-exec-10] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/runtime/streams'
2020-01-20 11:56:09.040 DEBUG 1 --- [io-8080-exec-10] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/runtime/apps'
2020-01-20 11:56:09.040 DEBUG 1 --- [io-8080-exec-10] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/runtime/apps/**'
2020-01-20 11:56:09.040 DEBUG 1 --- [io-8080-exec-10] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/streams/definitions'; against '/streams/definition
2020-01-20 11:56:09.040 DEBUG 1 --- [io-8080-exec-10] o.s.s.w.a.i.FilterSecurityInterceptor : Secure object: FilterInvocation: URL: /streams/definitions?page=0&size=30&sort=nC; Attributes: [hasRole('ROLE_VIEW')]
2020-01-20 11:56:09.040 DEBUG 1 --- [io-8080-exec-10] o.s.s.w.a.i.FilterSecurityInterceptor : Previously Authenticated: org.springframework.security.oauth2.client.authenticatuth2AuthenticationToken@7fc2f270: Principal: Name: [mk], Granted Authorities: [[ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW]], Useibutes: [{sub=40e9e738-a11c-4508-ab32-e1860ea74782, email_verified=false, user_name=mk, iss=http://keycloak-http:8844/auth/realms/scdf23, typ=ID, preferred_username=mk, nonce=iBV10Qo7JANF3EFJKaNCRyXJ-kzDCj3v8GMuQ, aud=[dataflow], acr=0, nbf=Thu Jan 01 01:00:00 CET 1970, azp=dataflow, auth_time=2020-01-20T10:23:58Z, exp=2020-01-20T10:54:01Z, session_sta51372-6f10-4a14-b742-d63484edae20, iat=2020-01-20T10:49:01Z, jti=e2adbd1c-3043-4b0b-960d-887f52ed1358}]; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframewcurity.web.authentication.WebAuthenticationDetails@fffdaa08: RemoteIpAddress: 127.0.0.1; SessionId: 31DC8D0782B881D13B8D45D251DDEA1E; Granted Authorities: ROLE_CREATE, ROLE_DEPLLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW
2020-01-20 11:56:09.040 DEBUG 1 --- [io-8080-exec-10] o.s.s.access.vote.AffirmativeBased : Voter: org.springframework.security.web.access.expression.WebExpressionVoter@4f4 returned: 1
2020-01-20 11:56:09.040 DEBUG 1 --- [io-8080-exec-10] o.s.s.w.a.i.FilterSecurityInterceptor : Authorization successful
2020-01-20 11:56:09.040 DEBUG 1 --- [io-8080-exec-10] o.s.s.w.a.i.FilterSecurityInterceptor : RunAsManager did not change Authentication object
2020-01-20 11:56:09.040 DEBUG 1 --- [io-8080-exec-10] o.s.security.web.FilterChainProxy : /streams/definitions?page=0&size=30&sort=name,ASC reached end of additional filtin; proceeding with original chain
2020-01-20 11:56:09.061 ERROR 1 --- [io-8080-exec-10] o.s.c.d.s.c.RestControllerAdvice : Caught exception while handling a request
org.springframework.web.client.HttpClientErrorException$Unauthorized: 401 Unauthorized
at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:81)
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:123)
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:102)
at org.springframework.cloud.skipper.client.SkipperClientResponseErrorHandler.handleError(SkipperClientResponseErrorHandler.java:78)
at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:785)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:743)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:698)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:625)
at org.springframework.cloud.skipper.client.DefaultSkipperClient.status(DefaultSkipperClient.java:133)
at org.springframework.cloud.dataflow.server.stream.SkipperStreamDeployer.getStreamDeploymentState(SkipperStreamDeployer.java:170)
at org.springframework.cloud.dataflow.server.stream.SkipperStreamDeployer.streamsStates(SkipperStreamDeployer.java:159)
at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService.state(DefaultStreamService.java:332)
at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService$$FastClassBySpringCGLIB$$89697014.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)More testing, I think that it is a problem related with connections, more exactly, kubectl port-forward I've made a mp4 video where you can see it, but size 100 mb cannot be uploaded to github. I am trying to make another ...
Scene is:
scdf and skipper start without problems with lastest configs, now i can create stream deployid ... very slow, but k8s dashboard , you can see the pods
force problem: restart port-forward tunnels.
and you can see that 401 appears into the network panel from browser and 401 appears checking any option in streaming
PD no runtime info is dsplayed ...
when kubectl port forward restart connection, scdf user is "anonymous" and skipper dont know what is a valid token
jvalkeal
commented
4 years ago 10.107.158.21:80 [/10.107.158.21] failed: Connection refused means nothing on that port, probably skipper was not yet running.
No deployer named 'default', you've probably named your account under kubernetes something else. If you didn't start with empty database and you've been playing around, stream there may have default platform name. Both dataflow/skipper will log created/configure platform during a startup.
eskuai
commented
4 years ago hi @jvalkeal
The first one, skippers is running, be sure... that is the connection is broken and 401, anonymous role user
2) deployer default, we changed with lastest config yaml and it is working , something related about processing parsing yaml ...
3) I am removing kubectl port ford-ward with socat ... i'll tell you
eskuai
commented
4 years ago When kubectl port forward restart resilent connection, skipper and scdf dont shared the token .or similiar ... more traces logs about user role anonymous .. .why ?
eskuai
commented
4 years ago Applying socat tunnels for keycloak and scdf2 ...
It works more time, but, random, the same problem
500 /about
auth2.core.OAuth2AuthenticationException
2020-01-20 13:21:35.972 DEBUG 1 --- [nio-7577-exec-4] .o.s.r.w.BearerTokenAuthenticationFilter : Authentication request for failed!
org.springframework.security.oauth2.core.OAuth2AuthenticationException: Provided token [eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJab1JGc0ZLVlQ4WHQ0ZlJSYzZUczRBT0N1LU9hVHdjdjFpMWpmQkZiQk1BIn0.eyJqdGkiOiJkNDZkYzk4Zi05OTI2LTQ0ZTYtYjU3NS1kMDVmM2E5NTNhODMiLCJleHAiOjE1Nzk1MjI3NjcsIm5iZiI6MCwiaWF0IjoxNTc5NTIyNDY3LCJpc3MiOiJodHRwOi8va2V5Y2xvYWstaHR0cDo4ODQ0L2F1dGgvcmVhbG1zL3NjZGYyMyIsImF1ZCI6ImFjY291bnQiLCJzdWIiOiI0MGU5ZTczOC1hMTFjLTQ1MDgtYWIzMi1lMTg2MGVhNzQ3ODIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJkYXRhZmxvdyIsIm5vbmNlIjoiU1UxWkVucDgycGg1eVk4d0k5WDRaTXdnRkI4YjJWb0YyVmstRHNJUWc2QSIsImF1dGhfdGltZSI6MTU3OTUxNTgzOCwic2Vzc2lvbl9zdGF0ZSI6IjM5ZDUxMzcyLTZmMTAtNGExNC1iNzQyLWQ2MzQ4NGVkYWUyMCIsImFjciI6IjAiLCJyZWFsbV9hY2Nlc3MiOnsicm9sZXMiOlsib2ZmbGluZV9hY2Nlc3MiLCJ1bWFfYXV0aG9yaXphdGlvbiJdfSwicmVzb3VyY2VfYWNjZXNzIjp7ImRhdGFmbG93Ijp7InJvbGVzIjpbInNjZGYtcm9sZSJdfSwiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJvcGVuaWQgZGF0YWZsb3cubWFuYWdlIGRhdGFmbG93LmNyZWF0ZSBkYXRhZmxvdy52aWV3IGRhdGFmbG93LmRlcGxveSBkYXRhZmxvdy5zY2hlZHVsZSBwcm9maWxlIGRhdGFmbG93Lm1vZGlmeSBkYXRhZmxvdy5kZXN0cm95IGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJ1c2VyX25hbWUiOiJtayIsInByZWZlcnJlZF91c2VybmFtZSI6Im1rIn0.ff8Ih5Ns1BlTxapsGssZ6gPN5-VK9761A1wBur5tC9K1mlb-1sy-FQcGEs5Rj31LFTb2Ol1ULwcWhTbLzRLq4kLlBf_pqG1wCf2HBTC_RnwIeDBNoAIBxt4OkvpHbLNp1irTf5zuGpgFtrbfXaNusfUlHhyQLnd1-rBiJDkNupge1_vmWn8Lv9ydjvdF-L4p0yadijJA7HL4Ky3ty3lbFxDQAcrJQKldL0Um_OySQB7w8SueGXRQhYSmuOKnX7pH5SRJ26nu02QwS31u7pXk9Upszic3F7Jdd0vAFDteKIZ_jf78nyyNZHh2QtTbw7wphLCdFRt5pnPhmF1ld5FC3Q] isn't active
at org.springframework.security.oauth2.server.resource.authentication.OpaqueTokenAuthenticationProvider.authenticate(OpaqueTokenAuthenticationProvider.java:99) ~[spring-security-oauth2-resource-server-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:175) ~[spring-security-core-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:124) ~[spring-security-oauth2-resource-server-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
why is there a lot of "anonymous" user logging in scdf ?
eskuai
commented
4 years ago 5 from=913 to=1447
GET /streams/definitions?page=0&size=30&sort=name,ASC HTTP/1.1\r
Host: scdf2-server-data-flow-server:8080\r
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0\r
Accept: application/json\r
Accept-Language: es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3\r
Accept-Encoding: gzip, deflate\r
Content-Type: application/json\r
DNT: 1\r
Connection: keep-alive\r
Referer: http://scdf2-server-data-flow-server:8080/dashboard/\r
Cookie: JSESSIONID=C3B182EED1BDF3775183D4F539DBBA91\r
Pragma: no-cache\r
Cache-Control: no-cache\r
\r
< 2020/01/20 13:35:15.786589 length=377 from=214586 to=214962
HTTP/1.1 500 \r
X-Content-Type-Options: nosniff\r
X-XSS-Protection: 1; mode=block\r
Cache-Control: no-cache, no-store, max-age=0, must-revalidate\r
Pragma: no-cache\r
Expires: 0\r
X-Frame-Options: DENY\r
Content-Type: application/json\r
Transfer-Encoding: chunked\r
Date: Mon, 20 Jan 2020 12:35:15 GMT\r
Connection: close\r
\r
38\r
[{"logref":"Unauthorized","message":"401 Unauthorized"}]\r
< 2020/01/20 13:35:15.787573 length=5 from=214963 to=214967
0\r
\rwhat is different from localhost and k8s ...
1) pods 2) http keep alive
how am i be surre that readness and liveness are working with security? could be that readness get 401 and broken session ?
2020-01-20 13:50:46.471 DEBUG 1 --- [nio-8080-exec-7] o.s.s.w.a.AnonymousAuthenticationFilter : Populated SecurityContextHolder with anonymous token: 'org.springframework.security.authentication.AnonymousAuthenticationToken@cef273b9: Principal: anonymousUser; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@ffff4c9c: RemoteIpAddress: 10.39.0.0; SessionId: null; Granted Authorities: ROLE_ANONYMOUS'
2020-01-20 13:50:46.471 DEBUG 1 --- [nio-8080-exec-7] o.s.security.web.FilterChainProxy : /management/health at position 15 of 17 in additional filter chain; firing Filter: 'SessionManagementFilter'
2020-01-20 13:50:46.472 DEBUG 1 --- [nio-8080-exec-7] o.s.security.web.FilterChainProxy : /management/health at position 16 of 17 in additional filter chain; firing Filter: 'ExceptionTranslationFilter'
2020-01-20 13:50:46.472 DEBUG 1 --- [nio-8080-exec-7] o.s.security.web.FilterChainProxy : /management/health at position 17 of 17 in additional filter chain; firing Filter: 'FilterSecurityInterceptor'
2020-01-20 13:50:46.472 DEBUG 1 --- [nio-8080-exec-7] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/management/health'; against '/actuator/**'
2020-01-20 13:50:46.472 DEBUG 1 --- [nio-8080-exec-7] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/management/health'; against '/management/**'
2020-01-20 13:50:46.472 DEBUG 1 --- [nio-8080-exec-7] o.s.s.w.a.i.FilterSecurityInterceptor : Secure object: FilterInvocation: URL: /management/health; Attributes**`
why SessionId: null; Granted Authorities: ROLE_ANONYMOUS'
anonymousUser; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@ffff4c9c: RemoteIpAddress: 10.39.0.0; SessionId: null; Granted Authorities: ROLE_ANONYMOUS'
eskuai
commented
4 years ago why runtime got nothing, there would be 6 entries, because, there is 6 streams deployed.
eskuai
commented
4 years ago Is it something about failed readness or liveness ping, that clears security tokens o similar?
Tx
2020-01-20 15:58:07.577 DEBUG 1 --- [nio-8080-exec-6] o.s.s.w.a.i.FilterSecurityInterceptor : Previously Authenticated: org.springframework.security.oauth2.client.authentication.OAuth2AuthenticationToken@aa3de510: Principal: Name: [mk], Granted Authorities: [[ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW]], User Attributes: [{sub=40e9e738-a11c-4508-ab32-e1860ea74782, email_verified=false, user_name=mk, iss=http://keycloak-http:8844/auth/realms/scdf23, typ=ID, preferred_username=mk, nonce=nfs_kaWxkKG5hvU04m1LVnJ5nazGUdMqqASlteXGRU8, aud=[dataflow], acr=1, nbf=Thu Jan 01 01:00:00 CET 1970, azp=dataflow, auth_time=2020-01-20T14:52:35Z, exp=2020-01-20T14:57:35Z, session_state=a2758cce-f248-40bb-b4b0-48c7e8437097, iat=2020-01-20T14:52:35Z, jti=3e108f6d-eeed-47e5-97ce-bf9101c9d41d}]; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@fffdaa08: RemoteIpAddress: 127.0.0.1; SessionId: D32F7866E7AEEFB54FBD89E105D82452; Granted Authorities: ROLE_CREATE, ROLE_DEPLOY, ROLE_DESTROY, ROLE_MANAGE, ROLE_MODIFY, ROLE_SCHEDULE, ROLE_VIEW
2020-01-20 15:58:07.577 DEBUG 1 --- [nio-8080-exec-6] o.s.s.access.vote.AffirmativeBased : Voter: org.springframework.security.web.access.expression.WebExpressionVoter@4138b17d, returned: 1
2020-01-20 15:58:07.577 DEBUG 1 --- [nio-8080-exec-6] o.s.s.w.a.i.FilterSecurityInterceptor : Authorization successful
2020-01-20 15:58:07.577 DEBUG 1 --- [nio-8080-exec-6] o.s.s.w.a.i.FilterSecurityInterceptor : RunAsManager did not change Authentication object
2020-01-20 15:58:07.577 DEBUG 1 --- [nio-8080-exec-6] o.s.security.web.FilterChainProxy : /streams/definitions/a123y reached end of additional filter chain; proceeding with original chain
2020-01-20 15:58:07.596 ERROR 1 --- [nio-8080-exec-6] o.s.c.d.s.c.RestControllerAdvice : Caught exception while handling a request
org.springframework.web.client.HttpClientErrorException$Unauthorized: 401 Unauthorized
at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:81)
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:123)
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:102)
at org.springframework.cloud.skipper.client.SkipperClientResponseErrorHandler.handleError(SkipperClientResponseErrorHandler.java:78)
at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:785)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:743)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:698)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:625)
at org.springframework.cloud.skipper.client.DefaultSkipperClient.status(DefaultSkipperClient.java:133)
sabbyanandan
commented
4 years ago Once again, there are far too many logs, screenshots, questions to grok here — I don't know where to begin.
From what I could understand, it seems you were able to run the standalone sample that we built (just for you), up and running. It also looks you can use the same sample to run it against your keycloak successfully.
If that setup is working, I was hoping you will be able to troubleshoot and figure how to get things up and running in K8s. It sounds like you couldn't. We are back at going back and forth in comments instead.
Here're some thoughts.
/configprops are loaded up correctly when SCDF/Skipper starts. Compare the /configprops results with the standalone setup. Determine what is different, fix them, and retry. You have a local working setup, and that is a great reference to compare configurations to get it working on other platforms.If none of this is working, we have no choice but finding time to invest and getting things working on our side against k8s+keycloak. Please also note that we aren't keycloak or k8s experts, but we can try, though. If it comes down to that, we will see what we can do once when the releases are complete next week.
eskuai
commented
4 years ago Hi @sabbyanandan
Sorry, it is my way to get a ledge ...
Whether local, k8s, or cf ... they all use the SAME binary; so same uber-jar; so same Spring Boot app. There's absolutely nothing special in how the SCDF/Skipper servers work on these environments.
My guess is that you're not passing the Spring Security properties correctly. I would copy-paste the same exact properties that you used in the standalone setup as opposed to converting it to uppercase and formatting them differently. I will say this again, these are Spring Security (aka: Spring Boot) properties. They are not SCDF specific properties. As far as the properties and the keycloak configurations are correctly loaded, the servers should work exactly the same as how they did locally!!
I think that the properties are passing right, but, then, if it is not, why can i get a error? scdf2 and skipper is starting ok. but, in a time, allways getting 401 ... I understand thtat properties are from spring security .. i dont forget that you are helpping me ... thank you again ...
/configprops , sorry, but i dont know where is this uri ... not /management
As summary:
1) local jars and keycloak, with yaml (without task and keeps alive http) => ok 2) local jars and keycloak under k8s (without task and keeps alive http) => ok 3) keycloak k8s and scdf2/skipper k8s (WITH TASK and keeps alive http) =>
Your help is greatly appreciated!
eskuai
commented
4 years ago More info. Changed tunnels from kubectl port-forward to direct socat tunnels. It seems that socat http trace s is more useful... 401 cames from readness ping
Server-config incluing now a permit-all rule, it was only into skipper config, some many test ...
More, be sure that your services port, as default 80, needed change to 8080 by keycloak checks uri host and port. Services ports same pods port, specially keycloak. Scdf Will need into task launcher. Next process.
All restarted and surprise!, scdf works at now without problem.
Tomorrow morning will check status again and commit and push final yaml files.
eskuai
commented
4 years ago gmt +2 8:00 it is working withou problems, no news .... 11:10 getting damm 401 11;10 tunnels is up, going to keycloak console 11:15 i don't know, there is 3 dataflow client sessions, i got the only one connection ... 1 ... 11:15 close all sessiones 11:15 restart tunnels, 11;:17 scdf2 redirect to kyecloak to as user and pass 11:17 login ok
These are different in dataflow vs. skipper. It should start with
spring.cloud.skipper.. Not yet sure if this is causing those errors.https://github.com/eskuai/scdf230keycloaktest/blob/771a84a20e5d6853fb8c6e0f8768845fa3ff35e6/skipper-deployment.yaml#L82