spring-cloud / spring-cloud-dataflow

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
https://dataflow.spring.io
Apache License 2.0
1.11k stars 583 forks source link

SCDF inside docker with enabled LDAP get 401 status from CTR when start a task #2871

Closed Rostish closed 5 years ago

Rostish commented 5 years ago

Starting SCDF and CTR in Docker Container with enabled LDAP.

Release versions: SCDf 1.7.3/1.7.4.RELEASE CTR 2.0.2.RELEASE

Custom apps: My own app that forms the DSL graph, creates and runs the task using dataFlowTemplate.

Steps to reproduce: 1) Start SCDF inside docker container with enabledLDAP auth. 2) Create and launch task, i tried with next parameters:

--spring.cloud.task.executionid: 3443 --dataflow-server-uri: http://10.101.48.150:9494 (9494 port is true for inside enviroment of conteiner the same for outside env) --increment-instance-enabled: true --dataflow-server-password: ** --dataflow-server-username: ** --spring.cloud.task.closecontext-enabled: true:

3) Inside logs of CTR you will see ERROR connected with status 401

Screenshots: image

Strange thing. I tried to remote debug CTR(downloaded latest ver from 2.0.2 branch). I used next parameters: image

but inside properties i don't see them, looks like it started without them image

inside SCDF logs i see my arguments too: image

Additional context: If i start SCDF locally on my remote server with enabledLDAP without any docker-compose it works fine. Problem appears just then it works with LDAP in DOCKER enviroment.

My Docker-Compose.file:

version: '3.3'
services:
  dataflow-multirating-server:
    build: /opt/databus_newfusion/scdf_multirating
    container_name: dataflow-multirating-server
    user: 1003:1001
    volumes:
      - /opt/databus_newfusion/scdf_multirating/task:/maven/tasks
      - /mnt/sas/data:/mnt/sas/data
    ports:
      - "9494:9494"
      - "17080:8080"
    expose:
      - "8077"
      - "9494"
    environment:
      JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
      spring.datasource.url: jdbc:postgresql://192.168.21.70:5432/data_flow
      spring.datasource.username: xxx
      spring.datasource.password: xxx
      spring.datasource.driver-class-name: org.postgresql.Driver
      server.port: 9494
      spring.cloud.dataflow.task.maximum-concurrent-tasks: 300
      spring.cloud.dataflow.security.authentication.ldap.enabled: "true"
      spring.cloud.dataflow.security.authentication.ldap.url: ldap://xxx.xxx:389
      spring.cloud.dataflow.security.authentication.ldap.managerDn: CN=xxxx,OU=xxxx,DC=xxx,DC=xxx
      spring.cloud.dataflow.security.authentication.ldap.managerPassword: xxxxx
      spring.cloud.dataflow.security.authentication.ldap.userSearchBase: dc=xxx,dc=xxx
      spring.cloud.dataflow.security.authentication.ldap.userSearchFilter: samaccountname={0}
      spring.cloud.dataflow.security.authentication.ldap.groupSearchBase: OU=xxx,DC=xxx,DC=xxx
      spring.cloud.dataflow.security.authentication.ldap.groupSearchFilter: member={0}
      spring.cloud.dataflow.security.authentication.ldap.roleMappings.ROLE_MANAGE: xxx
      spring.cloud.dataflow.security.authentication.ldap.roleMappings.ROLE_VIEW: xxx
      spring.cloud.dataflow.security.authentication.ldap.roleMappings.ROLE_CREATE: xxx
      security.basic.enabled: "true"
      security.baris.realm: 'Spring Cloud Data Flow'

  MultiratingRunnerSCDF:
    image: springcloud/openjdk:latest
    container_name: MultiratingRunnerSCDF
    expose:
      - "18765"
    user: 1003:1001
    ports:
      - "18755:18765"
      - "18788:18766"
      - "11024:8000"
    depends_on:
      - dataflow-multirating-server
    volumes:
      - /opt/databus_newfusion/scdf_multirating:/opt/databus_newfusion/scdf_multirating
      - /etc/localtime:/etc/localtime
    entrypoint: /bin/sh -c "sleep 30 && cd /opt/databus_newfusion/scdf_multirating && java -Dfile.encoding=UTF8 -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000 -Dspring.profiles.active=main-prod -jar /opt/databus_newfusion/scdf_multirating/schemerunner-develop-SNAPSHOT.jar"

  app-import:
    image: springcloud/openjdk:latest
    depends_on:
      - dataflow-multirating-server
    command: >
      /bin/sh -c "
        while ! nc -z dataflow-multirating-server 9494;
        do
          sleep 1;
        done;
        wget http://dataflow-multirating-server:9494/apps/task/multirating-baseoperation --post-data='uri=file://opt/tasks/baseoperation-1.0.1.jar';
        wget http://dataflow-multirating-server:9494/apps/task/composed-task-runner --post-data='uri=file://opt/tasks/composedtaskrunner-task-2.0.2.RELEASE.jar';
        echo 'Task apps imported'"
Rostish commented 5 years ago

I suppose i have found solution. @cppwfs gave me a hint about another problem there was an extra whitespace, i need to check it in prod enviroment. I will write here then i get results.

Rostish commented 5 years ago

i have checked. All work, close this ticket plz.

P.S. Don't know but maybe this is good idea to check extra whitespaces between lauch arguments, hard to find such problem. @cppwfs big ty for your hint in gitter.

sabbyanandan commented 5 years ago

Thanks for reporting it and the follow-up!