search

eclipse-tractusx / tractus-x-umbrella

Where all Tractus-X comes together
Apache License 2.0
12 stars 24 forks source link

chore: Update BPDM version #156

Closed CDiezRodriguez closed 1 week ago

CDiezRodriguez commented 3 weeks ago

Description

Update the BPDM version to 5.1.0 #140

Pre-review checks

Please ensure to do as many of the following checks as possible, before asking for committer review:

CDiezRodriguez commented 3 weeks ago

Looks really good! If you want you could also look into building back the workaround for the permission configuration discussed here. It should not be necessary anymore, because the CX-Central realm for the centralidp Keycloak should now contain all the configuration (relevant change eclipse-tractusx/portal-iam#155)

cc: @nicoprow

Hi @evegufy,

I think I’ll need to enable "Cl25-CX-BPDM-Orchestrator" to delete the permissions configuration since this client has the right permissions set up. Let me know if that sounds right, or if there’s anything else I should double-check with the new CX-Central realm setup in centralidp Keycloak.

image

evegufy commented 3 weeks ago

Looks really good! If you want you could also look into building back the workaround for the permission configuration discussed here. It should not be necessary anymore, because the CX-Central realm for the centralidp Keycloak should now contain all the configuration (relevant change eclipse-tractusx/portal-iam#155) cc: @nicoprow

Hi @evegufy,

I think I’ll need to enable "Cl25-CX-BPDM-Orchestrator" to delete the permissions configuration since this client has the right permissions set up. Let me know if that sounds right, or if there’s anything else I should double-check with the new CX-Central realm setup in centralidp Keycloak.

image

Hi @CDiezRodriguez thank you for look into this. @nicoprow could you please answer this?

evegufy commented 2 weeks ago

@nicoprow could you please provide feedback?

nicoprow commented 2 weeks ago

Looks really good! If you want you could also look into building back the workaround for the permission configuration discussed here. It should not be necessary anymore, because the CX-Central realm for the centralidp Keycloak should now contain all the configuration (relevant change eclipse-tractusx/portal-iam#155) cc: @nicoprow

Hi @evegufy,

I think I’ll need to enable "Cl25-CX-BPDM-Orchestrator" to delete the permissions configuration since this client has the right permissions set up. Let me know if that sounds right, or if there’s anything else I should double-check with the new CX-Central realm setup in centralidp Keycloak.

* [BPDM Doc](https://github.com/eclipse-tractusx/bpdm/blob/v6.1.0/docs/arc42/arc42-bpdm.md#permissions-as-client-resources)

* [BPDM Code](https://github.com/eclipse-tractusx/bpdm/blob/v6.1.0/bpdm-orchestrator/src/main/kotlin/org/eclipse/tractusx/bpdm/orchestrator/config/PermissionConfigProperties.kt#L29)

image

@CDiezRodriguez @evegufy It shouldn't be necessary to activate the Orchestrator Client as we just use it to define the Keycloak roles there. The Orchestrator client is not directly used for authentication but just a container for the role definitions. BPDM uses technical users (service accounts) that are separately defined in Keycloak for authentication.

Again, the BPDM Orchestrator references Cl25-CX-BPDM-Orchestrator to know where to look for roles in user token. Cl25-CX-BPDM-Orchestrator does not need to be active for this.

CDiezRodriguez commented 2 weeks ago

Hi @nicoprow, thanks for looking into this. @evegufy, at the moment, I can't delete the permissions because there is no client associated with the requested permissions. I'm not sure if we need to create a specific client for this.

evegufy commented 2 weeks ago

Looks really good! If you want you could also look into building back the workaround for the permission configuration discussed here. It should not be necessary anymore, because the CX-Central realm for the centralidp Keycloak should now contain all the configuration (relevant change eclipse-tractusx/portal-iam#155) cc: @nicoprow

Hi @evegufy, I think I’ll need to enable "Cl25-CX-BPDM-Orchestrator" to delete the permissions configuration since this client has the right permissions set up. Let me know if that sounds right, or if there’s anything else I should double-check with the new CX-Central realm setup in centralidp Keycloak.

* [BPDM Doc](https://github.com/eclipse-tractusx/bpdm/blob/v6.1.0/docs/arc42/arc42-bpdm.md#permissions-as-client-resources)

* [BPDM Code](https://github.com/eclipse-tractusx/bpdm/blob/v6.1.0/bpdm-orchestrator/src/main/kotlin/org/eclipse/tractusx/bpdm/orchestrator/config/PermissionConfigProperties.kt#L29)

image

@CDiezRodriguez @evegufy It shouldn't be necessary to activate the Orchestrator Client as we just use it to define the Keycloak roles there. The Orchestrator client is not directly used for authentication but just a container for the role definitions. BPDM uses technical users (service accounts) that are separately defined in Keycloak for authentication.

Again, the BPDM Orchestrator references Cl25-CX-BPDM-Orchestrator to know where to look for roles in user token. Cl25-CX-BPDM-Orchestrator does not need to be active for this.

@nicoprow My comment here was about building back the workaround in the helm values file, which shouldn't be needed anymore. My point was not about doing any change in the CX-Central realm config because that should now be as it's supposed to be. Could you please provide the input for building back the workaround in the helm values file for the (now resolved) permission issue?

nicoprow commented 2 weeks ago

Hi @nicoprow, thanks for looking into this. @evegufy, at the moment, I can't delete the permissions because there is no client associated with the requested permissions. I'm not sure if we need to create a specific client for this.

For 24.08. the portal-iam should contain the necessary clients in order to setup the golden record process. Here is the list of clients. Also all clients should have matching permissions with BPDM default roles. Therefore, you would need to remove the permission overrides and assign the new client credentials to the BPDM apps:

  1. Remove overrides in applicationProperties.bpdm.security.permissions for Orchestrator and Gate
  2. Assign clients for Cleaning Service Dummy, Pool and Gate as per table
App Client Configuration Client Name
Gate client.pool sa-cl7-cx-1
Gate client.orchestrator sa-cl25-cx-3
Cleaning Service Dummy client.orchestrator sa-cl25-cx-1
Pool client.orchestrator sa-cl25-cx-2
CDiezRodriguez commented 2 weeks ago

Hi @nicoprow and @evegufy, thank you for the support!

I followed @nicoprow’s instructions, but the BPDM deployment failed because the pool, gate, and orchestrator components could not connect to the database. Below is the relevant log:

2024-11-15 09:18:56.202 ERROR [System    ] [No Request] [main           ] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Exception during pool initialization.
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
 at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:342) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:54) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:263) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.Driver.makeConnection(Driver.java:443) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.Driver.connect(Driver.java:297) ~[postgresql-42.6.2.jar!/:42.6.2]
 at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:359) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:201) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:470) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:100) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112) ~[HikariCP-5.0.1.jar!/:na]
 at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:48) ~[flyway-core-9.22.3.jar!/:na]
 at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.<init>(JdbcConnectionFactory.java:74) ~[flyway-core-9.22.3.jar!/:na]
 at org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:142) ~[flyway-core-9.22.3.jar!/:na]
 at org.flywaydb.core.Flyway.migrate(Flyway.java:140) ~[flyway-core-9.22.3.jar!/:na]
 at org.springframework.boot.autoconfigure.flyway.FlywayMigrationInitializer.afterPropertiesSet(FlywayMigrationInitializer.java:66) ~[spring-boot-autoconfigure-3.2.5.jar!/:3.2.5]
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1833) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1782) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:600) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:522) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:313) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1234) ~[spring-context-6.1.6.jar!/:6.1.6]
 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:952) ~[spring-context-6.1.6.jar!/:6.1.6]
 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:624) ~[spring-context-6.1.6.jar!/:6.1.6]
 at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:456) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.springframework.boot.SpringApplication.run(SpringApplication.java:334) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.springframework.boot.SpringApplication.run(SpringApplication.java:1354) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.eclipse.tractusx.bpdm.pool.ApplicationKt.main(Application.kt:36) ~[!/:6.1.0]
 at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source) ~[na:na]
 at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na]
 at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91) ~[app.jar:6.1.0]
 at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53) ~[app.jar:6.1.0]
 at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58) ~[app.jar:6.1.0]
Caused by: java.net.ConnectException: Connection refused
 at java.base/sun.nio.ch.Net.pollConnect(Native Method) ~[na:na]
 at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source) ~[na:na]
 at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(Unknown Source) ~[na:na]
 at java.base/sun.nio.ch.NioSocketImpl.connect(Unknown Source) ~[na:na]
 at java.base/java.net.SocksSocketImpl.connect(Unknown Source) ~[na:na]
 at java.base/java.net.Socket.connect(Unknown Source) ~[na:na]
 at org.postgresql.core.PGStream.createSocket(PGStream.java:243) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.core.PGStream.<init>(PGStream.java:98) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:132) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:258) ~[postgresql-42.6.2.jar!/:42.6.2]
 ... 41 common frames omitted
2024-11-15 09:18:56.205 WARN  [System    ] [No Request] [main           ] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Unable to obtain connection from database: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State  : 08001
Error Code : 0
Message    : Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
2024-11-15 09:18:56.209 INFO  [System    ] [No Request] [main           ] o.apache.catalina.core.StandardService   : Stopping service [Tomcat]
2024-11-15 09:18:56.221 INFO  [System    ] [No Request] [main           ] .s.b.a.l.ConditionEvaluationReportLogger : 
Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2024-11-15 09:18:56.232 ERROR [System    ] [No Request] [main           ] o.s.boot.SpringApplication               : Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Unable to obtain connection from database: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State  : 08001
Error Code : 0
Message    : Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

After investigating, I noticed that the database service was missing. I added it as shown in this PR, and the deployment now works.

However, a new issue has arisen. The onboarding process fails at the BPN step because the pool, gate, and cleaning service components cannot communicate with the orchestrator. Here’s an excerpt from the logs:

24-11-15 09:23:00.002 ERROR [System    ] [No Request] [scheduling-1   ] o.e.t.b.c.service.CleaningServiceDummy   : Error while processing cleaning task
org.springframework.web.reactive.function.client.WebClientRequestException: finishConnect(..) failed: Connection refused: /[0:0:0:0:0:0:0:1]:80
 at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:136) ~[spring-webflux-6.1.6.jar!/:6.1.6]
 Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
 *__checkpoint ⇢ Request to POST /v6/golden-record-tasks/step-reservations [DefaultWebClient]

I suspect this might be happening because the components cannot locate the orchestrator. While checking the BPDM code, I found the following section in the application.yaml, which could be relevant.

Could you help confirm if I’m on the right track or suggest what else I might be missing? Thanks in advance!

Edit: I tried setting the orchestrator applicationConfig.bpdm.security.client-id to "Cl7-CX-BPDM", but it’s still not working.

Edit 2:

I just modified the orchestrator.base-url to point to the orchestrator service, but it didn’t work.

        client:
          orchestrator:
            base-url: http://umbrella-bpdm-orchestrator

I received a 403 response, although the orchestrator did receive the request.

Cleaning Service Log:

-11-15 11:12:30.000 INFO  [System    ] [No Request] [scheduling-1   ] o.e.t.b.c.service.CleaningServiceDummy   : Starting polling for cleaning tasks from Orchestrator... TaskStep CleanAndSync
2024-11-15 11:12:30.009 ERROR [System    ] [No Request] [scheduling-1   ] o.e.t.b.c.service.CleaningServiceDummy   : Error while processing cleaning task
org.springframework.web.reactive.function.client.WebClientResponseException$Forbidden: 403 Forbidden from POST http://umbrella-bpdm-orchestrator/v6/golden-record-tasks/step-reservations
 at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:309) ~[spring-webflux-6.1.6.jar!/:6.1.6]
 Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
 *__checkpoint ⇢ 403 FORBIDDEN from POST http://umbrella-bpdm-orchestrator/v6/golden-record-tasks/step-reservations [DefaultWebClient]

Orchestrator Log:

2024-11-15 11:12:30.005 INFO  [00003CRHK}] [c353fe62a1] [nio-8085-exec-7] o.e.t.b.common.config.UserLoggingFilter  : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks/step-reservations...
2024-11-15 11:12:30.006 INFO  [00003CRHK}] [f2985a5b91] [nio-8085-exec-8] o.e.t.b.common.config.UserLoggingFilter  : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks/step-reservations...
2024-11-15 11:12:30.008 INFO  [00003CRHK}] [415d87827b] [io-8085-exec-10] o.e.t.b.common.config.UserLoggingFilter  : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks/state/search...
2024-11-15 11:12:30.027 INFO  [00003CRHK}] [5c079af8b7] [nio-8085-exec-9] o.e.t.b.common.config.UserLoggingFilter  : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks...
2024-11-15 11:12:32.452 INFO  [Anonymous ] [0e322abd02] [nio-8085-exec-2] o.e.t.b.common.config.UserLoggingFilter  : User 'Anonymous' requests GET /actuator/health/liveness...
2024-11-15 11:12:32.452 INFO  [Anonymous ] [a132a3a62c] [nio-8085-exec-1] o.e.t.b.common.config.UserLoggingFilter  : User 'Anonymous' requests GET /actuator/health/readiness...
2024-11-15 11:12:32.453 INFO  [Anonymous ] [a132a3a62c] [nio-8085-exec-1] o.e.t.b.common.config.UserLoggingFilter  : Response with status 200
2024-11-15 11:12:32.453 INFO  [Anonymous ] [0e322abd02] [nio-8085-exec-2] o.e.t.b.common.config.UserLoggingFilter  : Response with status 200
evegufy commented 1 week ago

@nicoprow could you please have a look at the latest comment from @CDiezRodriguez and support?

nicoprow commented 1 week ago

Hi @nicoprow and @evegufy, thank you for the support!

I followed @nicoprow’s instructions, but the BPDM deployment failed because the pool, gate, and orchestrator components could not connect to the database. Below is the relevant log:

2024-11-15 09:18:56.202 ERROR [System    ] [No Request] [main           ] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Exception during pool initialization.
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
 at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:342) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:54) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:263) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.Driver.makeConnection(Driver.java:443) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.Driver.connect(Driver.java:297) ~[postgresql-42.6.2.jar!/:42.6.2]
 at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:359) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:201) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:470) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:100) ~[HikariCP-5.0.1.jar!/:na]
 at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112) ~[HikariCP-5.0.1.jar!/:na]
 at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:48) ~[flyway-core-9.22.3.jar!/:na]
 at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.<init>(JdbcConnectionFactory.java:74) ~[flyway-core-9.22.3.jar!/:na]
 at org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:142) ~[flyway-core-9.22.3.jar!/:na]
 at org.flywaydb.core.Flyway.migrate(Flyway.java:140) ~[flyway-core-9.22.3.jar!/:na]
 at org.springframework.boot.autoconfigure.flyway.FlywayMigrationInitializer.afterPropertiesSet(FlywayMigrationInitializer.java:66) ~[spring-boot-autoconfigure-3.2.5.jar!/:3.2.5]
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1833) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1782) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:600) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:522) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:313) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[spring-beans-6.1.6.jar!/:6.1.6]
 at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1234) ~[spring-context-6.1.6.jar!/:6.1.6]
 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:952) ~[spring-context-6.1.6.jar!/:6.1.6]
 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:624) ~[spring-context-6.1.6.jar!/:6.1.6]
 at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:456) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.springframework.boot.SpringApplication.run(SpringApplication.java:334) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.springframework.boot.SpringApplication.run(SpringApplication.java:1354) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343) ~[spring-boot-3.2.5.jar!/:3.2.5]
 at org.eclipse.tractusx.bpdm.pool.ApplicationKt.main(Application.kt:36) ~[!/:6.1.0]
 at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source) ~[na:na]
 at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na]
 at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91) ~[app.jar:6.1.0]
 at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53) ~[app.jar:6.1.0]
 at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58) ~[app.jar:6.1.0]
Caused by: java.net.ConnectException: Connection refused
 at java.base/sun.nio.ch.Net.pollConnect(Native Method) ~[na:na]
 at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source) ~[na:na]
 at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(Unknown Source) ~[na:na]
 at java.base/sun.nio.ch.NioSocketImpl.connect(Unknown Source) ~[na:na]
 at java.base/java.net.SocksSocketImpl.connect(Unknown Source) ~[na:na]
 at java.base/java.net.Socket.connect(Unknown Source) ~[na:na]
 at org.postgresql.core.PGStream.createSocket(PGStream.java:243) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.core.PGStream.<init>(PGStream.java:98) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:132) ~[postgresql-42.6.2.jar!/:42.6.2]
 at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:258) ~[postgresql-42.6.2.jar!/:42.6.2]
 ... 41 common frames omitted
2024-11-15 09:18:56.205 WARN  [System    ] [No Request] [main           ] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Unable to obtain connection from database: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State  : 08001
Error Code : 0
Message    : Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
2024-11-15 09:18:56.209 INFO  [System    ] [No Request] [main           ] o.apache.catalina.core.StandardService   : Stopping service [Tomcat]
2024-11-15 09:18:56.221 INFO  [System    ] [No Request] [main           ] .s.b.a.l.ConditionEvaluationReportLogger : 
Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2024-11-15 09:18:56.232 ERROR [System    ] [No Request] [main           ] o.s.boot.SpringApplication               : Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Unable to obtain connection from database: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State  : 08001
Error Code : 0
Message    : Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

After investigating, I noticed that the database service was missing. I added it as shown in this PR, and the deployment now works.

However, a new issue has arisen. The onboarding process fails at the BPN step because the pool, gate, and cleaning service components cannot communicate with the orchestrator. Here’s an excerpt from the logs:

24-11-15 09:23:00.002 ERROR [System    ] [No Request] [scheduling-1   ] o.e.t.b.c.service.CleaningServiceDummy   : Error while processing cleaning task
org.springframework.web.reactive.function.client.WebClientRequestException: finishConnect(..) failed: Connection refused: /[0:0:0:0:0:0:0:1]:80
 at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:136) ~[spring-webflux-6.1.6.jar!/:6.1.6]
 Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
 *__checkpoint ⇢ Request to POST /v6/golden-record-tasks/step-reservations [DefaultWebClient]

I suspect this might be happening because the components cannot locate the orchestrator. While checking the BPDM code, I found the following section in the application.yaml, which could be relevant.

Could you help confirm if I’m on the right track or suggest what else I might be missing? Thanks in advance!

Edit: I tried setting the orchestrator applicationConfig.bpdm.security.client-id to "Cl7-CX-BPDM", but it’s still not working.

Edit 2:

I just modified the orchestrator.base-url to point to the orchestrator service, but it didn’t work.

        client:
          orchestrator:
            base-url: http://umbrella-bpdm-orchestrator

I received a 403 response, although the orchestrator did receive the request.

Cleaning Service Log:

-11-15 11:12:30.000 INFO  [System    ] [No Request] [scheduling-1   ] o.e.t.b.c.service.CleaningServiceDummy   : Starting polling for cleaning tasks from Orchestrator... TaskStep CleanAndSync
2024-11-15 11:12:30.009 ERROR [System    ] [No Request] [scheduling-1   ] o.e.t.b.c.service.CleaningServiceDummy   : Error while processing cleaning task
org.springframework.web.reactive.function.client.WebClientResponseException$Forbidden: 403 Forbidden from POST http://umbrella-bpdm-orchestrator/v6/golden-record-tasks/step-reservations
 at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:309) ~[spring-webflux-6.1.6.jar!/:6.1.6]
 Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
 *__checkpoint ⇢ 403 FORBIDDEN from POST http://umbrella-bpdm-orchestrator/v6/golden-record-tasks/step-reservations [DefaultWebClient]

Orchestrator Log:

2024-11-15 11:12:30.005 INFO  [00003CRHK}] [c353fe62a1] [nio-8085-exec-7] o.e.t.b.common.config.UserLoggingFilter  : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks/step-reservations...
2024-11-15 11:12:30.006 INFO  [00003CRHK}] [f2985a5b91] [nio-8085-exec-8] o.e.t.b.common.config.UserLoggingFilter  : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks/step-reservations...
2024-11-15 11:12:30.008 INFO  [00003CRHK}] [415d87827b] [io-8085-exec-10] o.e.t.b.common.config.UserLoggingFilter  : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks/state/search...
2024-11-15 11:12:30.027 INFO  [00003CRHK}] [5c079af8b7] [nio-8085-exec-9] o.e.t.b.common.config.UserLoggingFilter  : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks...
2024-11-15 11:12:32.452 INFO  [Anonymous ] [0e322abd02] [nio-8085-exec-2] o.e.t.b.common.config.UserLoggingFilter  : User 'Anonymous' requests GET /actuator/health/liveness...
2024-11-15 11:12:32.452 INFO  [Anonymous ] [a132a3a62c] [nio-8085-exec-1] o.e.t.b.common.config.UserLoggingFilter  : User 'Anonymous' requests GET /actuator/health/readiness...
2024-11-15 11:12:32.453 INFO  [Anonymous ] [a132a3a62c] [nio-8085-exec-1] o.e.t.b.common.config.UserLoggingFilter  : Response with status 200
2024-11-15 11:12:32.453 INFO  [Anonymous ] [0e322abd02] [nio-8085-exec-2] o.e.t.b.common.config.UserLoggingFilter  : Response with status 200

Under normal circumstances the BPDM charts can infer the Postgres dependency name and the BPDM client service names directly from the given name and fullnameOverrides in the values. However, due to a bug this does not work when using this version of BPDM as a subchart. I was not aware that this bug is present in version 5.1.0.

But instead of letting the BPDM Charts infer the names you can just straight up directly set them, as you have already done. However, I propose these changes:

  1. I would not use the umbrella-names, as they will change based on the Chart release name. Instead, I recommend to give the BPDM postgres a fixed name via bpdm.postgres.fullnameOverride (and setting nameOverride blank) and use that name in each datasource.host property. By this, even if somebody changes the release name, the person does not need to change the values file.
  2. For the client base urls please use the ingress names (like http://business-partners.tx.test/pool) as they also do not change based on the Chart release name.

Regarding the 403 I just double checked the configuration. I forgot to mention that the Orchestrator's client-id (applicationProperties.bpdm.security.client-id) now needs to point to its own Keycloak client "Cl25-CX-BPDM-Orchestrator". Then it should work.

CDiezRodriguez commented 1 week ago

Thanks to @nicoprow's hints, it now works fine. I tested it successfully with the onboarding.