search

SolaceProducts / solace-spring-cloud

An umbrella project containing all Solace projects for Spring Cloud
Apache License 2.0
22 stars 15 forks source link

[SOL-57956] Do not include `spring-boot-starter-cloud-connectors` and `java-cfenv-boot` as compile / runtime dependencies #15

Closed FWinkler79 closed 2 years ago

FWinkler79 commented 4 years ago

We are running into major problems when the Spring Cloud Stream Binder Solace is on the classpath and our application is deployed to Cloud Foundry (using the standard java_buildpack). The issue is that IllegalStateExceptions are thrown at application startup due to the Cloud Connectors and java-cfenv libs being included in the application deployable.

The only way to fix this right now is to add Solace Binder like this:

    <dependency>
      <groupId>com.solace.spring.cloud</groupId>
      <artifactId>spring-cloud-starter-stream-solace</artifactId>
      <version>${spring.cloud.streams.solace.binder.version}</version>
      <exclusions>
        <exclusion>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter-cloud-connectors</artifactId>
        </exclusion>
        <exclusion>
          <groupId>io.pivotal.cfenv</groupId>
          <artifactId>java-cfenv-boot</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

Needless to say that it takes ages to find out what causes the issue and then concluding how to solve it.

The dependencies excluded above should either be removed from Solace Binder altogether or added as <optional> if necessary.

Nephery commented 4 years ago

Initial thoughts are that we probably need to update SolaceServiceAutoConfiguration to support java-cfenv instead of the deprecated cloud-connectors.

@FWinkler79 any chance you could provide stacktrace for the error you're getting?

Mrc0113 commented 4 years ago

Thanks @Nephery for looking into this!

@FWinkler79 if you have paid support can you please open a support ticket referring to the issues that you've opened? This can potentially help get them a higher priority. Also if you're part of the solace.community shoot me a message on there as I'd love to jump on a virtual chat and get to know more about what you're building, how you're using SCSt and/or other Solace libs, etc. and get your input on ways we can improve the Solace Developer experience :)

FWinkler79 commented 4 years ago

Hi @Nephery,

thank you for your reply. Here a stacktrace of a project that does not exclude spring-cloud-connectors and java-cfenv-boot.

2020-07-09T10:21:44.43+0200 [API/73] OUT Creating build for app with guid 3cd853b6-1d32-49ea-bf50-babae4d45de7
   2020-07-09T10:21:44.89+0200 [API/73] OUT Updated app with guid 3cd853b6-1d32-49ea-bf50-babae4d45de7 ({"state"=>"STARTED"})
   2020-07-09T10:21:45.09+0200 [STG/0] OUT Downloading java_buildpack...
   2020-07-09T10:21:45.10+0200 [STG/0] OUT Downloaded java_buildpack
   2020-07-09T10:21:45.10+0200 [STG/0] OUT Cell 17472d70-a6ae-4906-8c3d-6d3b1c8b6e95 creating container for instance fd9f20f2-9440-4d75-9f30-35a78dd83e45
   2020-07-09T10:21:45.84+0200 [STG/0] OUT Cell 17472d70-a6ae-4906-8c3d-6d3b1c8b6e95 successfully created container for instance fd9f20f2-9440-4d75-9f30-35a78dd83e45
   2020-07-09T10:21:45.98+0200 [STG/0] OUT Downloading app package...
   2020-07-09T10:21:47.72+0200 [STG/0] OUT Downloaded app package (60.8M)
   2020-07-09T10:21:49.27+0200 [STG/0] OUT -----> Java Buildpack v4.31.1 (offline) | https://github.com/cloudfoundry/java-buildpack.git#37b278c0
   2020-07-09T10:21:49.31+0200 [STG/0] OUT -----> Downloading Jvmkill Agent 1.16.0_RELEASE from https://java-buildpack.cloudfoundry.org/jvmkill/bionic/x86_64/jvmkill-1.16.0-RELEASE.so (found in cache)
   2020-07-09T10:21:49.31+0200 [STG/0] OUT -----> Downloading Open Jdk JRE 11.0.7_10 from https://java-buildpack.cloudfoundry.org/openjdk/bionic/x86_64/bellsoft-jre11.0.7%2B10-linux-amd64.tar.gz (found in cache)
   2020-07-09T10:21:50.43+0200 [STG/0] OUT        Expanding Open Jdk JRE to .java-buildpack/open_jdk_jre (1.1s)
   2020-07-09T10:21:50.43+0200 [STG/0] OUT        JVM DNS caching disabled in lieu of BOSH DNS caching
   2020-07-09T10:21:50.43+0200 [STG/0] OUT -----> Downloading Open JDK Like Memory Calculator 3.13.0_RELEASE from https://java-buildpack.cloudfoundry.org/memory-calculator/bionic/x86_64/memory-calculator-3.13.0-RELEASE.tar.gz (found in cache)
   2020-07-09T10:21:51.36+0200 [STG/0] OUT        Loaded Classes: 30176, Threads: 250
   2020-07-09T10:21:51.38+0200 [STG/0] OUT -----> Downloading Client Certificate Mapper 1.11.0_RELEASE from https://java-buildpack.cloudfoundry.org/client-certificate-mapper/client-certificate-mapper-1.11.0-RELEASE.jar (found in cache)
   2020-07-09T10:21:51.38+0200 [STG/0] OUT -----> Downloading Container Security Provider 1.18.0_RELEASE from https://java-buildpack.cloudfoundry.org/container-security-provider/container-security-provider-1.18.0-RELEASE.jar (found in cache)
   2020-07-09T10:21:51.38+0200 [STG/0] OUT -----> Downloading Spring Auto Reconfiguration 2.11.0_RELEASE from https://java-buildpack.cloudfoundry.org/auto-reconfiguration/auto-reconfiguration-2.11.0-RELEASE.jar (found in cache)
   2020-07-09T10:22:03.14+0200 [STG/0] OUT Exit status 0
   2020-07-09T10:22:03.14+0200 [STG/0] OUT Uploading droplet, build artifacts cache...
   2020-07-09T10:22:03.14+0200 [STG/0] OUT Uploading droplet...
   2020-07-09T10:22:03.14+0200 [STG/0] OUT Uploading build artifacts cache...
   2020-07-09T10:22:03.21+0200 [STG/0] OUT Uploaded build artifacts cache (130B)
   2020-07-09T10:22:04.52+0200 [API/106] OUT Creating droplet for app with guid 3cd853b6-1d32-49ea-bf50-babae4d45de7
   2020-07-09T10:22:09.62+0200 [STG/0] OUT Uploaded droplet (109.6M)
   2020-07-09T10:22:09.63+0200 [STG/0] OUT Uploading complete
   2020-07-09T10:22:09.84+0200 [STG/0] OUT Cell 17472d70-a6ae-4906-8c3d-6d3b1c8b6e95 stopping instance fd9f20f2-9440-4d75-9f30-35a78dd83e45
   2020-07-09T10:22:09.84+0200 [STG/0] OUT Cell 17472d70-a6ae-4906-8c3d-6d3b1c8b6e95 destroying container for instance fd9f20f2-9440-4d75-9f30-35a78dd83e45
   2020-07-09T10:22:10.45+0200 [CELL/0] OUT Cell f7300e8e-e754-4c22-8504-b79848132b06 creating container for instance dd71c641-bfc6-4b9d-7df4-0199
   2020-07-09T10:22:10.84+0200 [STG/0] OUT Cell 17472d70-a6ae-4906-8c3d-6d3b1c8b6e95 successfully destroyed container for instance fd9f20f2-9440-4d75-9f30-35a78dd83e45
   2020-07-09T10:22:11.14+0200 [CELL/0] OUT Cell f7300e8e-e754-4c22-8504-b79848132b06 successfully created container for instance dd71c641-bfc6-4b9d-7df4-0199
   2020-07-09T10:22:11.30+0200 [CELL/0] OUT Downloading droplet...
   2020-07-09T10:22:14.65+0200 [CELL/0] OUT Downloaded droplet (109.6M)
   2020-07-09T10:22:14.65+0200 [CELL/0] OUT Starting health monitoring of container
   2020-07-09T10:22:14.93+0200 [APP/PROC/WEB/0] OUT JVM Memory Configuration: -Xmx347488K -Xss1M -XX:ReservedCodeCacheSize=240M -XX:MaxDirectMemorySize=10M -XX:MaxMetaspaceSize=189087K
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT 09-07-2020 08:22:18.526 [main]  ERROR o.s.boot.SpringApplication.reportFailure - Application run failed
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT java.lang.IllegalStateException: Exiting the application since the Spring Cloud Connector library has been detected on the classpath.  Please remove this dependency from your project and set the environment variable JBP_CONFIG_SPRING_AUTO_RECONFIGURATION '{enabled: false}' in the Cloud Foundry manifest.
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at io.pivotal.cfenv.spring.boot.ConnectorLibraryDetector.assertNoConnectorLibrary(ConnectorLibraryDetector.java:43)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at io.pivotal.cfenv.spring.boot.CfDataSourceEnvironmentPostProcessor.postProcessEnvironment(CfDataSourceEnvironmentPostProcessor.java:78)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.context.config.ConfigFileApplicationListener.onApplicationEnvironmentPreparedEvent(ConfigFileApplicationListener.java:188)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.context.config.ConfigFileApplicationListener.onApplicationEvent(ConfigFileApplicationListener.java:176)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:127)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:76)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:53)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:345)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.SpringApplication.run(SpringApplication.java:308)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at com.sap.cloud.tenantlifecycle.euporie.service.Application.main(Application.java:11)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at java.base/java.lang.reflect.Method.invoke(Unknown Source)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
   2020-07-09T10:22:18.53+0200 [APP/PROC/WEB/0] OUT     at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
   2020-07-09T10:22:18.56+0200 [APP/PROC/WEB/0] OUT Exit status 1
   2020-07-09T10:22:19.07+0200 [CELL/0] OUT Cell f7300e8e-e754-4c22-8504-b79848132b06 stopping instance dd71c641-bfc6-4b9d-7df4-0199
   2020-07-09T10:22:19.07+0200 [CELL/0] OUT Cell f7300e8e-e754-4c22-8504-b79848132b06 destroying container for instance dd71c641-bfc6-4b9d-7df4-0199
   2020-07-09T10:22:19.09+0200 [API/41] OUT Process has crashed with type: "web"

This is the log of the startup of the application in CF. Note the IllegalStateException. Interesting fact: it seems this exception is thrown by the java-cfenv-boot dependencies (see also this issue) and it is gone as soon as not only spring-cloud-connectors but also java-cfenv-boot is excluded from the classpath.

What I was wondering anyway: solace binder includes both spring-cloud-connectors and java-cfenv-boot as compile dependencies. I always thought these are competing technologies. I was wondering if including both at the same time actually makes sense at all. But I could be mistaken there.

Cheers!

FWinkler79 commented 4 years ago

Thanks @Mrc0113,

I am working for a company that should have paid support. However, I have to get in touch with the colleagues that are in touch with you guys and have them open a ticket. So the communication would be kind of "proxied". If getting access to the Solace Community is free, I can open an account there and we get in touch. I can tell you more then.

Cheers!

P.S.: I created an account as FWinkler79, just ping me, if you'd like to discuss this further.

Nephery commented 4 years ago

Thanks @FWinkler79 . Looking at your logs, I'm now pretty much certain that what I said was the issue.

This is the log of the startup of the application in CF. Note the IllegalStateException. Interesting fact: it seems this exception is thrown by the java-cfenv-boot dependencies (see also this issue) and it is gone as soon as not only spring-cloud-connectors but also java-cfenv-boot is excluded from the classpath.

Have you set the environments on your app needed for java-cfenv? If not, that's likely why you're still getting that error even after excluding spring-cloud-connectors. Keep in mind that it might still break since we still need to update that one configuration class in the binder.

What I was wondering anyway: solace binder includes both spring-cloud-connectors and java-cfenv-boot as compile dependencies. I always thought these are competing technologies. I was wondering if including both at the same time actually makes sense at all. But I could be mistaken there.

No, what you're saying is correct. java-cfenv superseded spring-cloud-connectors and it explicitly throws an error (the one your getting) when both are present since, as you've said, it doesn't make sense to have both at the same time.

Awhile back, we had replaced the cloud connectors project with java-cfenv in version 4.0.0 of Solace Java Spring Boot which this binder built on top of. But our binder still needs to be updated to get rid of everything regarding spring-cloud-connectors.

FWinkler79 commented 4 years ago

Hi @Nephery,

thanks for your answer. Yes, when the exception occurred, the initial thing we did was set the environment variables you mentioned. But that has some negative side effects. One - harmless - side effect is, that the cloud profile is no longer automatically set. A more serious side effect, however, is that the CF buildpack no longer automatically "the bean definitions of various types to connect automatically with services bound to the application" (details here). This may have negative impact on database connections to DB services in CF etc. That's not something you want to lose, if you are running in CF.

Also, java-cfenv throws the exception because the java-buildpack adds the spring-cloud-connectors as part of the staging. We had excluded spring-cloud-connectors everywhere in our application, verified the effective pom.xml and even visually inspected all packaged dependencies in the Spring Boot uberjar - there was no trace of it left. Still at CF deployment, the exception occurred.

Generally, I would suggest to do two things:

  1. make both spring-cloud-connectors and java-cfenv optional dependencies, i.e. don't have your starter attract them and don't make them get packaged with the application. Let the application decide, which lib they want to use!
  2. Change your auto-configuration so that it can properly deal with both of them. Using the @ConditionalOnClass annotations, check which lib is on the classpath (because the app developer added it) and then initialize your beans accordingly.

That would allow app developers to make the choice that fits them best:

What do you think about that?

scottfrederick commented 4 years ago

As the maintainer of Spring Cloud Connectors and a contributor to Java CFEnv, I can provide some suggestions here. @dyroberts is the maintainer of Java CFEnv, and can also provide assistance if needed.

@Nephery said:

Awhile back, we had replaced the cloud connectors project with java-cfenv in version 4.0.0 of Solace Java Spring Boot which this binder built on top of. But our binder still needs to be updated to get rid of everything regarding spring-cloud-connectors.

This is the correct path to take. Spring Cloud Connectors is not being actively maintained, and its use is strongly discouraged in Spring Boot applications. The Connectors approach (which pre-dates Spring Boot) is to create and configure service connection beans and contribute them to the Spring application context, which competes with and is incompatible with Spring Boot auto-configuration. Over time, the connection configuration code in Connectors and the auto-configuration code in Spring Boot diverged, and it became costly to try to keep them in sync.

Java CFEnv was designed to address this by embracing auto-configuration and simply mapping Cloud Foundry VCAP_SERVICES information to Boot auto-configuration properties. This approach should be embraced by Solace Java Spring Boot.

@FWinkler79 said:

A more serious side effect, however, is that the CF buildpack no longer automatically "the bean definitions of various types to connect automatically with services bound to the application" (details here). This may have negative impact on database connections to DB services in CF etc. That's not something you want to lose, if you are running in CF.

Java Buildpack auto-reconfiguration is limited to a very small set of connection types, is not configurable or customizable, and has side-effects that can surprise users (mainly the fact that it looks for existing connection beans in an application context and replaces them with different connection beans). It was never intended to be used in production applications, but is more suited to a getting-started experience. Simply adding a dependency on Connectors (in the past) or Java CFEnv (in the present and future) provides all the benefits of auto-reconfiguration without the limitations, and is the recommended approach for production applications.

Also note that Java Buildpack auto-reconfiguration of connection beans backs off and does nothing if it detects that the application already contains the spring-cloud-connector libraries.

That said, there is an ongoing effort to update Java Buildpack auto-reconfiguration to support either Connectors or Java CFEnv for the getting-started experience. See https://github.com/cloudfoundry/java-buildpack/issues/779 and https://github.com/cloudfoundry/java-buildpack-auto-reconfiguration/pull/76 for discussion and status of this effort. This change should also make setting JBP_CONFIG_SPRING_AUTO_RECONFIGURATION '{enabled: false}' unnecessary, since the appropriate auto-reconfiguration will be chosen automatically.

Also, java-cfenv throws the exception because the java-buildpack adds the spring-cloud-connectors as part of the staging. We had excluded spring-cloud-connectors everywhere in our application, verified the effective pom.xml and even visually inspected all packaged dependencies in the Spring Boot uberjar - there was no trace of it left. Still at CF deployment, the exception occurred.

The spring-cloud-connectors dependency should not get added to the application classpath if JBP_CONFIG_SPRING_AUTO_RECONFIGURATION '{enabled: false}' is set as recommended in the documentation.

Generally, I would suggest to do two things:

  1. make both spring-cloud-connectors and java-cfenv optional dependencies, i.e. don't have your starter attract them and don't make them get packaged with the application. Let the application decide, which lib they want to use!
  2. Change your auto-configuration so that it can properly deal with both of them. Using the @ConditionalOnClass annotations, check which lib is on the classpath (because the app developer added it) and then initialize your beans accordingly.

As Spring Cloud Connectors is in maintenance mode, and its use is strongly discouraged in Spring Boot applications in favor of Java CFEnv, it would be better to remove all dependencies on spring-cloud-connectors instead of making it optional. There is no benefit to keeping Spring Cloud Connectors that can't be provided by and improved on by using Java CFEnv.

FWinkler79 commented 4 years ago

Great post @scottfrederick ! Thanks a lot for the clarifications.

My lessons learnt from your writings are:

Is that roughly correct?

Mrc0113 commented 4 years ago

Great post @scottfrederick ! Thanks a lot for the clarifications.

My lessons learnt from your writings are:

  • don't use CF java buildpack's auto-reconfiguration for production use, but rather switch it off and use java-cfenv instead.
  • hope for the buildpack to detect java-cfenv on the app's classpath automatically in the future and then back off gracefully without throwing exceptions.

Is that roughly correct?

Thanks so much for the info @scottfrederick! Just bumping @FWinkler79's response. Do you agree with his lessons learnt?

scottfrederick commented 4 years ago

@FWinkler79 @Mrc0113 Yes, I agree with both of those statements.