search

spring-cloud / spring-cloud-dataflow

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
https://dataflow.spring.io
Apache License 2.0
1.11k stars 581 forks source link

k8s scdf pod applying security user properties to run without root user #3469

Closed eskuai closed 5 years ago

eskuai commented 5 years ago

Description: As a user, I was to run scdf pod without root user ... Applying the k8s security options to run scdf using another using not root .. by example 1001. The scdf pod start running, but, when scdf try to read info about mapping maven configuration shows info about cannot read file system, because, the owner is root. Cannot import apps ...

Release versions: Scdf 2.2.0 skipper 2.1.0

Custom apps: Any app that needs not root user to run.

Steps to reproduce: Appply k8s config security properties to the configuration pod to run scdf without root, by example 1001 as user and group Failed to read filesystem

sabbyanandan commented 5 years ago

Hi, @eskuai. I'm not sure I fully understand what you're reporting here. Could you describe with a use-case? The more the details, the better it'd be helping to answer it.

eskuai commented 5 years ago

Hi @sabbyanandan , thank you for you help.

Applying the k8s security options to run scdf , we defined scdf deployment as:

dapiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ template "scdf.fullname" . }}-server
  labels:
    app: {{ template "scdf.name" . }}
    component: server
    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
    release: "{{ .Release.Name }}"
    heritage: "{{ .Release.Service }}"
spec:
  selector:
    matchLabels:
      app: {{ template "scdf.name" . }}
      component: server
      release: "{{ .Release.Name }}"
  replicas: 1
  template:
    metadata:
      labels:
        app: {{ template "scdf.name" . }}
        component: server
        release: "{{ .Release.Name }}"
    spec:
      securityContext:
        runAsUser: 1000
        runAsGroup: 3000
      containers:
      - name: {{ template "scdf.fullname" . }}-server
        image: {{ .Values.server.image }}:{{ .Values.server.version }}
        imagePullPolicy: {{ .Values.server.imagePullPolicy }}
        volumeMounts:

We check the pod ..

groups: cannot find name for group ID 3000
I have no name!@scdf2-data-flow-server-564dfc968c-5m4jz:/$ id
uid=1000 gid=3000 groups=3000
I have no name!@scdf2-data-flow-server-564dfc968c-5m4jz:/$

Um.. something about groups ... ok, forget about it ... i'll be back in another moment. ... Ok, let's go and try to deploy a stream,, time to log ... pods are configure to user 1000 Scdf pod shows a problem Unable to create directory for local repository: myLocalrepoMK About creating directory from maven local repository (we are using nexus), maybe, pod filesystem owner is root, but, scdf process id is 1000 ... cannot write into filesystem ???

2019-09-03 16:02:58.600 DEBUG 1 --- [nio-8080-exec-7] o.s.orm.jpa.JpaTransactionManager        : Initiating transaction rollback
2019-09-03 16:02:58.600 DEBUG 1 --- [nio-8080-exec-7] o.s.orm.jpa.JpaTransactionManager        : Rolling back JPA transaction on EntityManager [SessionImpl(406222643<open>)]
2019-09-03 16:02:58.600 DEBUG 1 --- [nio-8080-exec-7] o.h.e.t.internal.TransactionImpl         : rolling back
2019-09-03 16:02:58.601 DEBUG 1 --- [nio-8080-exec-7] o.s.orm.jpa.JpaTransactionManager        : Closing JPA EntityManager [SessionImpl(406222643<open>)] after transaction
2019-09-03 16:02:58.602 ERROR 1 --- [nio-8080-exec-7] o.s.c.d.s.c.RestControllerAdvice         : Caught exception while handling a request
org.springframework.cloud.deployer.resource.support.ResourceNotResolvedException: Unable to create directory for local repository: myLocalrepoMK
        at org.springframework.cloud.deployer.resource.support.DelegatingResourceLoader.getResource(DelegatingResourceLoader.java:92)
        at org.springframework.cloud.dataflow.registry.support.AppResourceCommon.getMetadataResource(AppResourceCommon.java:260)
        at org.springframework.cloud.dataflow.registry.service.DefaultAppRegistryService.getAppMetadataResource(DefaultAppRegistryService.java:271)
        at org.springframework.cloud.dataflow.registry.service.DefaultAppRegistryService$$FastClassBySpringCGLIB$$a8bae4.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
        at org.springframework.cloud.dataflow.registry.service.DefaultAppRegistryService$$EnhancerBySpringCGLIB$$8e9e2432.getAppMetadataResource(<generated>)
        at org.springframework.cloud.dataflow.server.service.impl.AppDeploymentRequestCreator.createRequests(AppDeploymentRequestCreator.java:193)
        at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService.doDeployStream(DefaultStreamService.java:150)
        at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService.deployStream(DefaultStreamService.java:426)
        at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService$$FastClassBySpringCGLIB$$89697014.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
        at org.springframework.cloud.dataflow.server.service.impl.DefaultStreamService$$EnhancerBySpringCGLIB$$45a301c2.deployStream(<generated>)
        at org.springframework.cloud.dataflow.server.controller.StreamDeploymentController.deploy(StreamDeploymentController.java:183)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1039)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:103)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:88)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.oauth2.provider.authentication.OAuth2AuthenticationProcessingFilter.doFilter(OAuth2AuthenticationProcessingFilter.java:176)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:74)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
        at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.security.oauth2.client.filter.OAuth2ClientContextFilter.doFilter(OAuth2ClientContextFilter.java:60)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:114)
        at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:104)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:853)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1587)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Unable to create directory for local repository: myLocalrepoMK
        at org.springframework.util.Assert.isTrue(Assert.java:118)
        at org.springframework.cloud.deployer.resource.maven.MavenArtifactResolver.<init>(MavenArtifactResolver.java:110)
        at org.springframework.cloud.deployer.resource.maven.MavenResource.<init>(MavenResource.java:122)
        at org.springframework.cloud.deployer.resource.maven.MavenResource.parse(MavenResource.java:268)
        at org.springframework.cloud.deployer.resource.maven.MavenResourceLoader.getResource(MavenResourceLoader.java:62)
        at org.springframework.cloud.deployer.resource.support.DelegatingResourceLoader.getResource(DelegatingResourceLoader.java:89)
        ... 133 common frames omitted
2019-09-03 16:02:58.603 DEBUG 1 --- [nio-8080-exec-7] o.apache.coyote.http11.Http11Processor   : Socket: [org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@577288a8:org.apache.tomcat.util.net.NioChannel@5dab716d:java.nio.channels.SocketChannel[connected local=/127.0.0.1:8080 remote=/127.0.0.1:57300]], Status in: [OPEN_READ], State out: [CLOSED]

and we got the problem into \"maven\": { \"local-repository\": \"myLocalrepoMK\", \"remote-repositories\": but, scdf config is :

         value: 'https://uaa-svc:8443/oauth/authorize'
        - name: SECURITY_OAUTH2_RESOURCE_USER_INFO_URI
          value: 'https://uaa-svc:8443/userinfo'
        - name: SECURITY_OAUTH2_RESOURCE_TOKEN_INFO_URI
          value: 'https://uaa-svc:8443/check_token'
        - name: SPRING_APPLICATION_JSON
          value: "{ \"javax.net.ssl.trustStore\": \"/tmp/scdf2cacerts/cacerts\",\"javax.net.ssl.trustStorePassword\": \"changeit\" , \"com.sun.net.ssl.checkRevocation\": \"false\", \"maven\": { \"local-repository\": \"myLocalrepoMK\", \"remote-repositories\": { \"mk-repository\": {\"url\": \"http://${NEXUS_SERVICE_HOST}:${NEXUS_SERVICE_PORT}/repository/maven-releases/\",\"auth\": {\"username\": \"admin\",\"password\": \"admin123\"}},\"spring-repo\": {\"url\": \"https://repo.spring.io/libs-release\",\"auth\": {\"username\": \"\",\"password\": \"\"}},\"spring-repo-snapshot\": {\"url\": \"https://repo.spring.io/libs-snapshot/\",\"auth\": {\"username\": \"\",\"password\": \"\"}}}} }"
      volumes:
        - name: tmpcacerts
          secret:

do we need a local maven repo?

sabbyanandan commented 5 years ago

Hi, @eskuai. Based on your update, it appears you're attempting to provision SCDF on K8s, and to do that, you have a fork of the SCDF Helm chart with customizations in it. Did I get this right?

Also, if your target platform is K8s, you must use the Docker images. In other words, you don't need to configure the Maven artifactory. You'd have to register the Docker image URIs of the apps that we ship and the ones that you're creating - more info here.

eskuai commented 5 years ago

Hi, @sabbyanandan. We are using SCDF on K8s, applying helm or manual deploying. boths are working ...

SCDF is running using Docker images from a docker registry. It is working ok, no problem.

Then, into the process registration for an app, we need: imagen type, uri, metadata ---> string, docker uri,... maven repo jar uri???` What uri can we set to a docker image and how create it to get metada uri working?

Now, we apply to referer a metadata uri, and it is a maven repository uri (nexus).

About scdf documentation, we can read:

By default, the protocol is set to http. You can omit the auth properties if the proxy does not need a username and password. Also, the maven localRepository is set to ${user.home}/.m2/repository/ by default.

Ok, let's apply runAsUser 1000,

spec:
     securityContext:
       runAsUser: 1000
       runAsGroup: 3000
       fsGroup: 2000

imagen

Then, an scdf process will write into "local directory" some info.

org.springframework.cloud.deployer.resource.support.ResourceNotResolvedException: Unable to create directory for local repository: ?/.m2/repository
Caused by: java.lang.IllegalArgumentException: Unable to create directory for local repository: ?/.m2/repository

And not working ... we get back to root user ...

what am i doing wrong?

Thank you

ilayaperumalg commented 5 years ago

Hi @eskuai, What happens when you don't set the metadata URI above? The metadata URI is optional and if you can't have a maven artifact as the metadata resource, you can leave it empty as the metadata resource with Docker artifact is not supported.

eskuai commented 5 years ago

Hi @ilayaperumalg Yes, but using only docker registering we cannot get the info properties for the app. How can i get then the properties from app?

ilayaperumalg commented 5 years ago

using only docker registering we cannot get the info properties for the app. How can i get then the properties from app?

If you register docker resource for the app and you want to have the properties for the app, the only way to do it is via an explicit maven artifact for the app metadata.

eskuai commented 5 years ago

We need all info about properties apps. Cannot we remove the config about maven local repository into scdf deployment? do we need explicity a "localrepository"? How can i configure the pid scdf pod user a nonroot? Because, i can mount a nfs pv volumen and mapping users into all nodes and pods into k8s cluster, and all users can allow to write into the disk. is it about .m2 default directorty? is it about new directory for maven in all nodes-pods? using nfs-pv with all permission allowed?

sabbyanandan commented 5 years ago

Hi, @eskuai. Thanks for the details.

do we need explicity a "localrepository"?

If you want to review the supported app properties, yes, it needs to be a maven artifact. And, that artifact needs to come from an artifactory.

How can i configure the pid scdf pod user a nonroot?

I don't have enough background in regards to pod's root user vs. process user. Honestly, I don't know the difference here. I'm going to request @chrisjs to have a look; he may have some insights to share.

All the rest of the questions that you listed above somewhat relates to the previous question being answered. So, let's start with that first.

chrisjs commented 5 years ago 
Caused by: java.lang.IllegalArgumentException: Unable to create directory for local repository: ?/.m2/repository
        - name: JAVA_TOOL_OPTIONS
          value: "-Duser.home=/tmp"

in the appropriate deployment file env block.

how you choose to configure this is based on your requirements. below is an example diff against src/kubernetes/server/server-deployment.yaml of using the UID/GID of nobody and setting the java system parameter for user.home to /tmp which would be accessible by anybody:

diff --git a/src/kubernetes/server/server-deployment.yaml b/src/kubernetes/server/server-deployment.yaml
index 00002d46..c1d40be1 100644
--- a/src/kubernetes/server/server-deployment.yaml
+++ b/src/kubernetes/server/server-deployment.yaml
@@ -14,6 +14,9 @@ spec:
       labels:
         app: scdf-server
     spec:
+      securityContext:
+        runAsUser: 65534
+        fsGroup: 65534
       containers:
       - name: scdf-server
         image: springcloud/spring-cloud-dataflow-server:2.3.0.BUILD-SNAPSHOT
@@ -33,7 +36,7 @@ spec:
             fieldRef:
               fieldPath: "metadata.namespace"
         - name: SERVER_PORT
-          value: '80'
+          value: '8080'
         - name: SPRING_CLOUD_CONFIG_ENABLED
           value: 'false'
         - name: SPRING_CLOUD_DATAFLOW_FEATURES_ANALYTICS_ENABLED
@@ -54,6 +57,8 @@ spec:
           # Add Maven repo for metadata artifact resolution for all stream apps
         - name: SPRING_APPLICATION_JSON
           value: "{ \"maven\": { \"local-repository\": null, \"remote-repositories\": { \"repo1\": { \"url\": \"https://repo.spring.io/libs-snapshot\"} } } }"
+        - name: JAVA_TOOL_OPTIONS
+          value: "-Duser.home=/tmp"
       initContainers:
       - name: init-mysql-wait
         image: busybox

NOTE: this also changes the server_port value to 8080 as non-root users cannot create ports under 1024. as you start making changes for non-root, you may have to change other things as well.

this is not a dataflow issue, more of how you choose to configure your security settings and get everything working properly. the above should get you started.

eskuai commented 5 years ago

Hi @chrisjs

if the ? is an indicator something like say ${user.home} couldn't be resolved, the UID your using likely has no HOME environment set (invalid user, no home dir for user, etc.) This is the "copy-paste" log message from scdf error ... i don't know why it used ... it could be more user-friendly ... it could says some more info into the log, apart of "?" ...it could be more usefull, " there is not defined a valid user using from configuration <?> to process the maven execution" .. some help please.. And the uid from docker running (1000, 1001) have got home user, because, there is "docker" user.

when an environment variable is attempted to be used but not resolvable, that results in an empty string, likely leaving the path /.m2/repository I dont know, this environment variable is for scdf into the properties app process. It is defined for you. I am asking if i can change, remove or what to do .. but i need to see properties from app

the error message is telling you that it can't create the directory, seemingly /.m2/repository. if you are using non-root ID's, obviously this is intended behavior as you would not have the proper permissions to do so as a non-root user Be sure, that i checked it, more one time, and the k8s pvc is not read only and it is mapping into pvc with a nfs directory with 777, and, watch out, the owner is 1000, into the master, with name "docker"...

realistically you should be using a UID/GID that exists in the typical /etc/passwd and /etc/group files of the container. I dont agree with it, Uid, guid from host is not passed to the guest ... more anonymous, more security. My "vision" is from administration Unix system(ci/cd) and I can hear from security docker and k8s groups it is not a best practice, and i've read a lot about linux docker, virtualization and security and we can talk about it in another moment (i think is very interesting). I am testing "podman" from RedHat to avoid all these problems. But i think this is not the forum neither problem.

``if you cannot do so, or the UID/GID you select does not have a HOME environment variable, you may have to do something like:

how you choose to configure this is based on your requirements. below is an example diff against src/kubernetes/server/server-deployment.yaml of using the UID/GID of nobody and setting the java system parameter for user.home to /tmp which would be accessible by anybody ... Again, from Unix process, we can not use -nobody- from security policies. This is the same problem like why scdf is using pid 1 for running java -jar .. um.... kubernetes 1.17 will fix early ... 1 pid into running proces Unix is not like the rest ... problems about signal, gargabe into docker process... and more ... sorry. .. this is another concern. "tiny proxy meantime"

Ok, i'll change java_tools to config user.home and i'll check if the metadata info for properties from apps will be save into this directory, but configuration applying /tmp" into a possible "restarted" process not sound best way.

But if works, dont touch

If you reopen the issue, i'll post the info ...

Thank you

chrisjs commented 5 years ago

the above, for example using nobody was a suggestion. i don’t know your environment so i can only comment in general terms.

if your getting permission errors, unresolved system parameters etc, you will need to review and adjust your configuration to match your requirements

eskuai commented 5 years ago

Hi @chrisjs

Right, we'll check it applying "tmp" place ... but, be sure, applying root with the same config is all working without problem .. and maven directories are 777 permission (I am tearing your hair out)

if i try a another pod running spring hello wolrd java reading from the same directory, damn, it is work .. and it doesn need a "tmp" environment ...

no way, i'll try applying an java_opts...

tx you again

chrisjs commented 5 years ago

again, that is an example using stock configs (except security ctx), in an environment where there are no restrictions on users, additional things like nfs mounts etc:

the settings you choose to use are based on your environment and needs.

psudhakar commented 3 years ago

Dear @eskuai , just wanted to check how di you resolve this issue ? I have run into same issue and I cannot run my pods under root. In my case, I am not even reading anything from meta properties file. Every time I run Composed Task, it fails with this error. Can you pls advise ?