search

spring-cloud / spring-cloud-dataflow

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
https://dataflow.spring.io
Apache License 2.0
1.11k stars 580 forks source link

AzureAD OAuth not working on Cloud Foundry #4572

Closed tommparekh closed 3 years ago

tommparekh commented 3 years ago

Hi, I have deployed SCDF on Cloud Foundry (Open Source implementation). I got my dataflow server dashboard setup as a client application in Azure AD and got client secret and other required end points. When I am trying to setup data flow server with Azure AD OAuth configuration per below link, I am seeing my application not starting in CF.

https://github.com/spring-cloud/spring-cloud-dataflow/blob/b6d094924d86fff3530cf83aefe6530f8ce10a55/spring-cloud-dataflow-docs/src/main/asciidoc/appendix-identity-provider-azure.adoc

My goal is to authenticate (& authorize) enterprise users while accessing data flow dashboard to ensure only authorized users can access dashboard functionalities.

Here is my application.properties for data flow server:

info:
  app:
    name: "spring-cloud-dataflow-server"
    description: "Spring Cloud Data Flow Server"
    version: "2.7.1"
server:
#  port: 8443
  ssl:
    enabled: true
    key-alias: dataflow.company.net
    key-store: /home/vcap/app/BOOT-INF/classes/dataflow-keystore.jks
    key-store-type: jks
    key-store-password: test
    key-password: test
    trust-store: /home/vcap/app/BOOT-INF/classes/dataflow-keystore.jks                 
    trust-store-password: test
# Security (Oauth 2):

spring:
  cloud:
    dataflow:
      security:
        authorization:
          provider-role-mappings:
            dataflow-server:
              map-oauth-scopes: true
              role-mappings:
                ROLE_VIEW: dataflow.view
                ROLE_CREATE: dataflow.create
                ROLE_MANAGE: dataflow.manage
                ROLE_DEPLOY: dataflow.deploy
                ROLE_DESTROY: dataflow.destroy
                ROLE_MODIFY: dataflow.modify
                ROLE_SCHEDULE: dataflow.schedule
  security:
    oauth2:
      client:
        registration:
          dataflow-server:
            provider: azure
            redirect-uri: '{baseUrl}/login/oauth2/code/{registrationId}'  # (should this be this exact URL or not?)
            client-id: 49266f4f-7sdfd-4977-95sfdsf-fdsfd77f43
            client-secret: sdf23gq3~02K-fdg3-x-PWn
#            authorization-grant-type: authorization_code
            scope:
            - openid
            - profile
            - email
            - offline_access
            - api://dataflow-server/dataflow.view
            - api://dataflow-server/dataflow.deploy
            - api://dataflow-server/dataflow.destroy
            - api://dataflow-server/dataflow.manage
            - api://dataflow-server/dataflow.modify
            - api://dataflow-server/dataflow.schedule
            - api://dataflow-server/dataflow.create
        provider:
          azure:
            issuer-uri: https://login.microsoftonline.com/jfiee84k-ereer-4f65-bretet2d-ertertw/v2.0
            user-name-attribute: name
      resourceserver:
        jwt:
          jwk-set-uri: https://azuread-proxy.company.com/jfiee84k-ereer-4f65-bretet2d-ertertw/discovery/v2.0/keys

Any help is appreciated.

sabbyanandan commented 3 years ago

@tommparekh: Any reason why you're not using the SCDF tile? The Tile internally integrates with UAA/SSO, so they have a cleaner integration with Azure AD, perhaps, and it is automated as opposed to manual configuration.

tommparekh commented 3 years ago

@sabbyanandan SCDF tile is not available in open source cloud foundry implementation at my site.

sabbyanandan commented 3 years ago

Okay, it is hard to tell what is missing, the version in use, or the errors you're getting. If you have a reproducible case with errors/logs, and the exact manifest.yml files in use, we can review it. Otherwise, we will be assuming too much.

tommparekh commented 3 years ago

Ok. I can provide you the information you will need to troubleshoot. Current state is, my application is deployed successfully and running. But when I hit application url (mapped route with/out /dashboard context path), I get below error. Also, I dont see the URL asking for authentication with Azure. No log entries suggesting the server will be using OAuth with Azure.

Bad Request This combination of host and port requires TLS.

Is there a working example of properties file for Azure AD integration with SCDF server on Cloud Foundry? Not sure if I am missing anything in config? I also dont see any errors in logs.

I have attached logs with the update. Please let me know what other details you may need from my side.


13:21:36.237: [CELL.0] Starting health monitoring of container
13:21:36.345: [APP/PROC/WEB.0] JVM Memory Configuration: -Xmx1397927K -Xss1M -XX:ReservedCodeCacheSize=240M -XX:MaxDirectMemorySize=10M -XX:MaxMetaspaceSize=187224K
13:21:38.674: [APP/PROC/WEB.0] 2021-06-10 17:21:38.669  INFO 14 --- [kground-preinit] o.h.validator.internal.util.Version      : HV000001: Hibernate Validator 6.1.6.Final
13:21:38.775: [APP/PROC/WEB.0] 2021-06-10 17:21:38.774  INFO 14 --- [           main] s.b.CfDataSourceEnvironmentPostProcessor : Setting spring.datasource properties from bound service [postgresql]
13:21:38.775: [APP/PROC/WEB.0] 2021-06-10 17:21:38.775  INFO 14 --- [           main] i.p.c.s.boot.CfEnvironmentPostProcessor  : Setting spring.data.mongodb properties from bound service [mongodb] using io.pivotal.cfenv.spring.boot.MongoCfEnvProcessor
13:21:38.775: [APP/PROC/WEB.0] 2021-06-10 17:21:38.775  INFO 14 --- [           main] i.p.c.s.boot.CfEnvironmentPostProcessor  : Setting spring.rabbitmq properties from bound service [rabbitmq] using io.pivotal.cfenv.spring.boot.AmqpCfEnvProcessor
13:21:39.514: [APP/PROC/WEB.0] 2021-06-10 17:21:39.511  WARN 14 --- [           main] io.fabric8.kubernetes.client.Config      : Error reading service account token from: [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring.
13:21:39.522: [APP/PROC/WEB.0] 2021-06-10 17:21:39.516  WARN 14 --- [           main] io.fabric8.kubernetes.client.Config      : Error reading service account token from: [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring.
13:21:39.522: [APP/PROC/WEB.0] 2021-06-10 17:21:39.518  WARN 14 --- [           main] o.s.c.k.KubernetesAutoConfiguration      : No namespace has been detected. Please specify KUBERNETES_NAMESPACE env var, or use a later kubernetes version (1.3 or later)
13:21:39.566: [APP/PROC/WEB.0] 2021-06-10 17:21:39.563  INFO 14 --- [           main] ContainerTrustManagerFactory$PKIXFactory : Adding System Trust Manager
13:21:39.749: [APP/PROC/WEB.0]   ____                              ____ _                __
13:21:39.749: [APP/PROC/WEB.0]  / ___| _ __  _ __(_)_ __   __ _   / ___| | ___  _   _  __| |
13:21:39.749: [APP/PROC/WEB.0]  \___ \| '_ \| '__| | '_ \ / _` | | |   | |/ _ \| | | |/ _` |
13:21:39.749: [APP/PROC/WEB.0]   ___) | |_) | |  | | | | | (_| | | |___| | (_) | |_| | (_| |
13:21:39.749: [APP/PROC/WEB.0]  |____/| .__/|_|  |_|_| |_|\__, |  \____|_|\___/ \__,_|\__,_|
13:21:39.749: [APP/PROC/WEB.0]   ____ |_|    _          __|___/                 __________
13:21:39.749: [APP/PROC/WEB.0]  |  _ \  __ _| |_ __ _  |  ___| | _____      __  \ \ \ \ \ \
13:21:39.749: [APP/PROC/WEB.0]  | | | |/ _` | __/ _` | | |_  | |/ _ \ \ /\ / /   \ \ \ \ \ \
13:21:39.749: [APP/PROC/WEB.0]  | |_| | (_| | || (_| | |  _| | | (_) \ V  V /    / / / / / /
13:21:39.749: [APP/PROC/WEB.0]  |____/ \__,_|\__\__,_| |_|   |_|\___/ \_/\_/    /_/_/_/_/_/
13:21:39.749: [APP/PROC/WEB.0] Spring Cloud Data Flow Server  (v2.7.1)
13:21:39.791: [APP/PROC/WEB.0] 2021-06-10 17:21:39.790  INFO 14 --- [           main] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://localhost:8888
13:21:39.907: [APP/PROC/WEB.0] 2021-06-10 17:21:39.906  INFO 14 --- [           main] c.c.c.ConfigServicePropertySourceLocator : Connect Timeout Exception on Url - http://localhost:8888. Will be trying the next url if available
13:21:39.907: [APP/PROC/WEB.0] 2021-06-10 17:21:39.907  WARN 14 --- [           main] c.c.c.ConfigServicePropertySourceLocator : Could not locate PropertySource: I/O error on GET request for "http://localhost:8888/data-flow-server/cloud": Connection refused (Connection refused); nested exception is java.net.ConnectException: Connection refused (Connection refused)
13:21:39.975: [APP/PROC/WEB.0] 2021-06-10 17:21:39.974  INFO 14 --- [           main] b.c.PropertySourceBootstrapConfiguration : Located property source: [BootstrapPropertySource {name='bootstrapProperties-configmap.data-flow-server.null'}]
13:21:40.027: [APP/PROC/WEB.0] 2021-06-10 17:21:40.026  INFO 14 --- [           main] o.s.c.d.s.s.DataFlowServerApplication    : The following profiles are active: cloud
13:21:41.266: [APP/PROC/WEB.0] 2021-06-10 17:21:41.265  INFO 14 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Multiple Spring Data modules found, entering strict repository configuration mode!
13:21:41.266: [APP/PROC/WEB.0] 2021-06-10 17:21:41.265  INFO 14 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data Map repositories in DEFAULT mode.
13:21:41.516: [APP/PROC/WEB.0] 2021-06-10 17:21:41.515  INFO 14 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 237ms. Found 1 Map repository interfaces.
13:21:42.089: [APP/PROC/WEB.0] 2021-06-10 17:21:42.088  INFO 14 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Multiple Spring Data modules found, entering strict repository configuration mode!
13:21:42.104: [APP/PROC/WEB.0] 2021-06-10 17:21:42.103  INFO 14 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 8ms. Found 0 JPA repository interfaces.
13:21:42.091: [APP/PROC/WEB.0] 2021-06-10 17:21:42.090  INFO 14 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
13:21:42.311: [APP/PROC/WEB.0] 2021-06-10 17:21:42.310  INFO 14 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Multiple Spring Data modules found, entering strict repository configuration mode!
13:21:42.311: [APP/PROC/WEB.0] 2021-06-10 17:21:42.310  INFO 14 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
13:21:42.365: [APP/PROC/WEB.0] 2021-06-10 17:21:42.364  INFO 14 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 53ms. Found 5 JPA repository interfaces.
13:21:42.557: [APP/PROC/WEB.0] 2021-06-10 17:21:42.556  INFO 14 --- [           main] o.s.cloud.context.scope.GenericScope     : BeanFactory id=0197a892-cdfd-3ed8-8f9f-4beb6818dc44
13:21:43.270: [APP/PROC/WEB.0] 2021-06-10 17:21:43.269  INFO 14 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (https)
13:21:43.287: [APP/PROC/WEB.0] 2021-06-10 17:21:43.286  INFO 14 --- [           main] o.a.coyote.http11.Http11NioProtocol      : Initializing ProtocolHandler ["https-jsse-nio-8080"]
13:21:43.287: [APP/PROC/WEB.0] 2021-06-10 17:21:43.287  INFO 14 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
13:21:43.288: [APP/PROC/WEB.0] 2021-06-10 17:21:43.287  INFO 14 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.41]
13:21:43.385: [APP/PROC/WEB.0] 2021-06-10 17:21:43.384  INFO 14 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
13:21:43.954: [APP/PROC/WEB.0] 2021-06-10 17:21:43.953  INFO 14 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
13:21:44.180: [APP/PROC/WEB.0] 2021-06-10 17:21:44.177  INFO 14 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
13:21:44.250: [APP/PROC/WEB.0] 2021-06-10 17:21:44.249  INFO 14 --- [           main] o.f.c.internal.license.VersionPrinter    : Flyway Community Edition 6.4.4 by Redgate
13:21:44.299: [APP/PROC/WEB.0] 2021-06-10 17:21:44.298  INFO 14 --- [           main] o.f.c.internal.database.DatabaseFactory  : Database: jdbc:postgresql://sdf-sdf-e8c2-4d6e-8174-sdf.apps.company.net:3232/postgres (PostgreSQL 11.9)
13:21:44.351: [APP/PROC/WEB.0] 2021-06-10 17:21:44.350  INFO 14 --- [           main] o.f.core.internal.command.DbValidate     : Successfully validated 4 migrations (execution time 00:00.021s)
13:21:44.360: [APP/PROC/WEB.0] 2021-06-10 17:21:44.360  INFO 14 --- [           main] o.f.core.internal.command.DbMigrate      : Current version of schema "public": 4
13:21:44.361: [APP/PROC/WEB.0] 2021-06-10 17:21:44.361  INFO 14 --- [           main] o.f.core.internal.command.DbMigrate      : Schema "public" is up to date. No migration necessary.
13:21:44.514: [APP/PROC/WEB.0] 2021-06-10 17:21:44.514  INFO 14 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
13:21:44.584: [APP/PROC/WEB.0] 2021-06-10 17:21:44.583  INFO 14 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate ORM core version 5.4.25.Final
13:21:44.743: [APP/PROC/WEB.0] 2021-06-10 17:21:44.742  INFO 14 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
13:21:44.946: [APP/PROC/WEB.0] 2021-06-10 17:21:44.945  INFO 14 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.PostgreSQL10Dialect
13:21:46.468: [APP/PROC/WEB.0] 2021-06-10 17:21:46.467  INFO 14 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
13:21:46.478: [APP/PROC/WEB.0] 2021-06-10 17:21:46.477  INFO 14 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
13:21:46.881: [APP/PROC/WEB.0] 2021-06-10 17:21:46.881  INFO 14 --- [           main] d.c.r.ContainerRegistryAutoConfiguration : Final Registry Configurations: {registry-1.docker.io=ContainerRegistryConfiguration{registryHost='registry-1.docker.io', user='null', secret='****'', authorizationType=dockeroauth2, manifestMediaType='application/vnd.docker.distribution.manifest.v2+json', disableSslVerification='false', useHttpProxy='false', extra={registryAuthUri=https://auth.docker.io/token?service=registry.docker.io&scope=repository:{repository}:pull&offline_token=1&client_id=shell}}}
13:21:47.886: [APP/PROC/WEB.0] 2021-06-10 17:21:47.885  WARN 14 --- [           main] .s.c.d.s.s.i.TaskConfigurationProperties : org.springframework.cloud.dataflow.server.service.impl.TaskConfigurationProperties.setComposedTaskRunnerUri is deprecated. Please use org.springframework.cloud.dataflow.server.service.impl.ComposedTaskRunnerConfigurationProperties.setUri
13:21:48.248: [APP/PROC/WEB.0] 2021-06-10 17:21:48.247  INFO 14 --- [           main] ContainerTrustManagerFactory$PKIXFactory : Adding System Trust Manager
13:21:48.259: [APP/PROC/WEB.0] 2021-06-10 17:21:48.259  INFO 14 --- [           main] ContainerTrustManagerFactory$PKIXFactory : Adding System Trust Manager
13:21:48.269: [APP/PROC/WEB.0] 2021-06-10 17:21:48.268  INFO 14 --- [           main] ContainerTrustManagerFactory$PKIXFactory : Adding System Trust Manager
13:21:48.277: [APP/PROC/WEB.0] 2021-06-10 17:21:48.276  INFO 14 --- [           main] ContainerTrustManagerFactory$PKIXFactory : Adding System Trust Manager
13:21:48.282: [APP/PROC/WEB.0] 2021-06-10 17:21:48.281  INFO 14 --- [           main] ContainerTrustManagerFactory$PKIXFactory : Adding System Trust Manager
13:21:48.317: [APP/PROC/WEB.0] 2021-06-10 17:21:48.317  INFO 14 --- [           main] ContainerTrustManagerFactory$PKIXFactory : Adding System Trust Manager
13:21:48.392: [APP/PROC/WEB.0] 2021-06-10 17:21:48.392  INFO 14 --- [           main] ContainerTrustManagerFactory$PKIXFactory : Adding System Trust Manager
13:21:48.518: [APP/PROC/WEB.0] 2021-06-10 17:21:48.517  WARN 14 --- [           main] cloudfoundry-client.trust                : Trusting SSL Certificate for api.cf.company.net:443
13:21:49.323: [APP/PROC/WEB.0] 2021-06-10 17:21:49.322  WARN 14 --- [-client-epoll-1] cloudfoundry-client.trust                : Trusting SSL Certificate for login.cf.company.net:443
13:21:50.061: [APP/PROC/WEB.0] 2021-06-10 17:21:50.060  INFO 14 --- [-client-epoll-1] .d.s.c.c.CloudFoundryTaskPlatformFactory : Connecting to Cloud Foundry with API Version 2.124.0
13:21:50.315: [APP/PROC/WEB.0] 2021-06-10 17:21:50.314  INFO 14 --- [           main] o.s.b.c.r.s.JobRepositoryFactoryBean     : No database type set, using meta data indicating: POSTGRES
13:21:50.367: [APP/PROC/WEB.0] 2021-06-10 17:21:50.366  INFO 14 --- [           main] o.s.c.d.s.b.SimpleJobServiceFactoryBean  : No database type set, using meta data indicating: POSTGRES
13:21:50.503: [APP/PROC/WEB.0] 2021-06-10 17:21:50.502  WARN 14 --- [           main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
13:21:50.610: [APP/PROC/WEB.0] 2021-06-10 17:21:50.609  INFO 14 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
13:21:51.552: [APP/PROC/WEB.0] 2021-06-10 17:21:51.551  INFO 14 --- [           main] ContainerTrustManagerFactory$PKIXFactory : Adding System Trust Manager
13:21:51.554: [APP/PROC/WEB.0] 2021-06-10 17:21:51.553  INFO 14 --- [           main] oundryContainerKeyManagerFactory$SunX509 : Adding System Key Manager
13:21:51.554: [APP/PROC/WEB.0] 2021-06-10 17:21:51.554  INFO 14 --- [           main] oundryContainerKeyManagerFactory$SunX509 : Adding Key Manager for /etc/cf-instance-credentials/instance.key and /etc/cf-instance-credentials/instance.crt
13:21:51.556: [APP/PROC/WEB.0] 2021-06-10 17:21:51.556  INFO 14 --- [-instance.crt-0] org.cloudfoundry.security.FileWatcher    : Start watching /etc/cf-instance-credentials/instance.crt
13:21:51.556: [APP/PROC/WEB.0] 2021-06-10 17:21:51.556  INFO 14 --- [-instance.key-0] org.cloudfoundry.security.FileWatcher    : Start watching /etc/cf-instance-credentials/instance.key
13:21:51.569: [APP/PROC/WEB.0] 2021-06-10 17:21:51.568  INFO 14 --- [           main] o.c.s.FileWatchingX509ExtendedKeyManager : Initialized KeyManager for /etc/cf-instance-credentials/instance.key and /etc/cf-instance-credentials/instance.crt
13:21:51.950: [APP/PROC/WEB.0] 2021-06-10 17:21:51.949  INFO 14 --- [           main] .s.c.DataFlowControllerAutoConfiguration : Skipper URI [http://skipper-server.cf.company.net/api]
13:21:52.244: [APP/PROC/WEB.0] 2021-06-10 17:21:52.243  INFO 14 --- [           main] o.a.coyote.http11.Http11NioProtocol      : Starting ProtocolHandler ["https-jsse-nio-8080"]
13:21:52.272: [APP/PROC/WEB.0] 2021-06-10 17:21:52.271  INFO 14 --- [           main] ContainerTrustManagerFactory$PKIXFactory : Adding System Trust Manager
13:21:52.299: [APP/PROC/WEB.0] 2021-06-10 17:21:52.298  INFO 14 --- [           main] oundryContainerKeyManagerFactory$SunX509 : Adding System Key Manager
13:21:52.302: [APP/PROC/WEB.0] 2021-06-10 17:21:52.301  INFO 14 --- [           main] ContainerTrustManagerFactory$PKIXFactory : Adding System Trust Manager
13:21:52.344: [APP/PROC/WEB.0] 2021-06-10 17:21:52.344  INFO 14 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (https) with context path ''
13:21:52.366: [APP/PROC/WEB.0] 2021-06-10 17:21:52.365  INFO 14 --- [           main] o.s.c.d.s.s.DataFlowServerApplication    : Started DataFlowServerApplication in 15.247 seconds (JVM running for 16.015)
13:21:52.581: [APP/PROC/WEB.0] 2021-06-10 17:21:52.580  INFO 14 --- [           main] .s.c.d.s.s.LauncherInitializationService : Added 'Cloud Foundry' platform account 'default' into Task Launcher repository.
13:21:52.619: [CELL.0] Container became healthy

Thank you for helping.

tommparekh commented 3 years ago

I am able to setup OAuth on CF now. We can close this issue. Thank you for your support.

As a solution, I am using OAuth setup from Spring security module.