search

mulesoft / mule-maven-plugin

Maven plugins for Mule Runtime
35 stars 45 forks source link

mule-deployer - JdkConnectorProvider does not support http/s proxy connections #813

Closed doodo477 closed 3 months ago

doodo477 commented 4 months ago

Hi.

I'm the lead integration developer at Charles Darwin University - CDU. We have recently been upgrading our Mulesoft app's to the new MuleSoft Java 17 version which has resulted in having to upgrade the Mulesoft Mule-Maven-Plugin from version 3.8.2 to 4.1.0. Our app migration are going fine how-ever we've run into problems when using our onprem gitlab runner to deploy the apps to the cloudhub, I've traced the problem to the (mule-maven-plugin\mule-deployer\src\main\java\org\mule\tools\client\core\AbstractClient.java ) file that set's the jersey.client.ClientConfig connectorProvider to org.glassfish.jersey.jdk.connector.JdkConnectorProvider instead of letting jersey use the default HttpUrlConnectorProvider.

Because of our local network security all of our on-prem docker containers must use a http/s proxy to connect to the internet. All requests from the docker container from the container host is blocked. Our gitlab CI script that we use to deploy our scripts is provided in [i]. Investigating the issues further, we found out that the JdkConnectorProvider does not support http/s proxies which are set via the https.proxyHost and https.proxyPort arguments. The jstack call stack of the problem is provided in [ii].

I've tested the following jersey Connector implementation's bellow in [iii], which are supported by jersey (https://eclipse-ee4j.github.io/jersey.github.io/documentation/latest/appendix-properties.html).

jersey.config.client.proxy.uri URI of a HTTP proxy the client connector should use. Default value is not set. Currently supported with ApacheConnectorProvider, Apache5ConnectorProvider, GrizzlyConnectorProvider, HelidonConnectorProvider, NettyConnectorProvider, and JettyConnectorProvider only.

How-ever, due to them modifying the http headers it results in a http response of 400 from cloudhub with the message org.jvnet.mimepull.MIMEParsingException: Missing start boundary.. This is also reported by other projects such as (https://github.com/eclipse-ee4j/jersey/issues/5094).

To solve our deployment problem I've removed the offending line configuration.connectorProvider(new org.glassfish.jersey.jdk.connector.JdkConnectorProvider()); from the (mule-maven-plugin\mule-deployer\src\main\java\org\mule\tools\client\core\AbstractClient.java ), and deployed a quickfix mule-maven-plugin 4.1.0-SNAPSHOT version to our gitlab repository, which has resolved the issue. But this isn't really a long term solution.

[iii] jersey Connector implementation's ApacheConnectorProvider GrizzlyConnectorProvider JettyConnectorProvider NettyConnectorProvider

[i] Gitlab .gitlab-ci.yml container script.

image: maven:3.9.6-eclipse-temurin-17

variables:
  APP_NAME: "student-admission-process-api"
  BUSINESSGROUP: "Student Services"
  MAVEN_CLI_OPTS: "-s $CI_BUILDS_DIR/.m2/settings.xml --batch-mode"
  MAVEN_OPTS: "-Dmaven.repo.local=/.m2/repository"

cache:
  key: one-key-to-rule-them-all
  when: 'always'
  paths:
    - .m2/repository

stages:
  - intialize
  - deploy
  - muleDeploy_dev
  - muleDeploy_prd

maven_intialize:
  stage: intialize
  only:
  - develop
  - JDK17upgrade
  - staging
  - master
  script:
    - echo "Copy Maven seting file"
    - mkdir -p $CI_BUILDS_DIR/.m2 && cp $MAVEN_SETTINGS_XML $CI_BUILDS_DIR/.m2/settings.xml
    - echo $CI_PROJECT_DIR
    - echo $PWD

maven_deploy_to_exchange:
  stage: deploy
  only:
  - develop
  - JDK17upgrade
  - staging
  script:
    - echo "Deploying the project (to Exchange) with the Mule Maven Plugin"
    - mvn $MAVEN_CLI_OPTS clean deploy

maven_deploy_to_dev:
  stage: muleDeploy_dev
  environment:
    name: DEV
    action: stop
  only:
  - develop
  - JDK17upgrade
  dependencies:
    - maven_deploy_to_exchange
  script:
  - echo "Deploying the project (to Cloudhub - DEV environment) with the Mule Maven Plugin"
  - mvn $MAVEN_CLI_OPTS deploy 
    -P cloudhub
    -U
    -DmuleDeploy
    -Dmule.applicationName=$APP_NAME
    -Dch.businessGroup="$BUSINESSGROUP"
    -Dch.connectedAppClientId=$ANYPOINT_CONNECTEDAPP_CLIENTID
    -Dch.connectedAppClientSecret=$ANYPOINT_CONNECTEDAPP_SECRET
    -Dmule.org.client_id=$ORGANIZATION_CLIENTID
    -Dmule.org.client_secret=$ORGANIZATION_SECRET
    -Dch.workers=1
    -Dch.workerType=MICRO
    -Dhttps.proxyHost=baza.cdu.edu.au
    -Dhttps.proxyPort=3128
    -Dmule.env.uri=dev
    -Dmule.env=DEV
    -Dmule.environment=DEV
    -Drtf.jsonlogger.priority=INFO
    -Dsecure.key="$SECURE_KEY_DEV"

[ii] Jstack when using the JdkConnectorProvider.

   java.lang.Thread.State: WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@17.0.10/Native Method)
        - parking to wait for  <0x00000000e1cf79e8> (a java.util.concurrent.CountDownLatch$Sync)
        at java.util.concurrent.locks.LockSupport.park(java.base@17.0.10/LockSupport.java:211)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@17.0.10/AbstractQueuedSynchronizer.java:715)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(java.base@17.0.10/AbstractQueuedSynchronizer.java:1047)
        at java.util.concurrent.CountDownLatch.await(java.base@17.0.10/CountDownLatch.java:230)
        at org.glassfish.jersey.jdk.connector.internal.ChunkedBodyOutputStream.doInitialBlocking(ChunkedBodyOutputStream.java:249)
        at org.glassfish.jersey.jdk.connector.internal.ChunkedBodyOutputStream.write(ChunkedBodyOutputStream.java:92)
        at org.glassfish.jersey.jdk.connector.internal.IntercecommitStreamptingOutputStream.write(InterceptingOutputStream.java:48)
        at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200)
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276)
        at sun.nio.cs.StreamEncoder.writeBytes(java.base@17.0.10/StreamEncoder.java:234)
        at sun.nio.cs.StreamEncoder.implFlushBuffer(java.base@17.0.10/StreamEncoder.java:313)
        at sun.nio.cs.StreamEncoder.implFlush(java.base@17.0.10/StreamEncoder.java:318)
        at sun.nio.cs.StreamEncoder.flush(java.base@17.0.10/StreamEncoder.java:160)
        - locked <0x00000000e1b90088> (a java.io.OutputStreamWriter)
        at java.io.OutputStreamWriter.flush(java.base@17.0.10/OutputStreamWriter.java:248)
        at org.glassfish.jersey.message.internal.ReaderWriter.writeToAsString(ReaderWriter.java:170)
        at org.glassfish.jersey.message.internal.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:107)
        at org.glassfish.jersey.message.internal.StringMessageProvider.writeTo(StringMessageProvider.java:76)
        at org.glassfish.jersey.message.internal.StringMessageProvider.writeTo(StringMessageProvider.java:36)
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242)
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227)
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
        at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116)
        at org.glassfish.jersey.client.ClientRequest.doWriteEntity(ClientRequest.java:461)
        at org.glassfish.jersey.client.ClientRequest.writeEntity(ClientRequest.java:443)
        at org.glassfish.jersey.jdk.connector.internal.JdkConnector.apply(JdkConnector.java:128)
        at org.glassfish.jersey.jdk.connector.internal.JdkConnector.apply(JdkConnector.java:59)
        at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:297)
        at org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$0(JerseyInvocation.java:662)
        at org.glassfish.jersey.client.JerseyInvocation$$Lambda$2676/0x00007fddf0a42000.call(Unknown Source)
        at org.glassfish.jersey.client.JerseyInvocation.call(JerseyInvocation.java:697)
        at org.glassfish.jersey.client.JerseyInvocation.lambda$runInScope$3(JerseyInvocation.java:691)
        at org.glassfish.jersey.client.JerseyInvocation$$Lambda$2677/0x00007fddf0a42240.call(Unknown Source)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:205)
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
        at org.glassfish.jersey.client.JerseyInvocation.runInScope(JerseyInvocation.java:691)
        at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:661)
        at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:439)
        at org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:345)
        at org.mule.tools.client.core.AbstractClient.post(AbstractClient.java:61)
        at org.mule.tools.client.authentication.AuthenticationServiceClient.loginWithConnectedApp(AuthenticationServiceClient.java:147)
        at org.mule.tools.client.authentication.AuthenticationServiceClient.getBearerTokenForConnectedApp(AuthenticationServiceClient.java:88)
        at org.mule.tools.client.AbstractMuleClient.getBearerToken(AbstractMuleClient.java:340)
        at org.mule.tools.client.AbstractMuleClient.init(AbstractMuleClient.java:100)
        at org.mule.tools.client.core.AbstractClient.initialize(AbstractClient.java:125)
        - locked <0x00000000e3ef6bf0> (a org.mule.tools.client.cloudhub.CloudHubClient)
        at org.mule.tools.client.core.AbstractClient.get(AbstractClient.java:90)
        at org.mule.tools.client.cloudhub.CloudHubClient.isDomainAvailable(CloudHubClient.java:199)
        at org.mule.tools.deployment.cloudhub.CloudHubArtifactDeployer.createOrUpdateApplication(CloudHubArtifactDeployer.java:123)
        at org.mule.tools.deployment.cloudhub.CloudHubArtifactDeployer.deployApplication(CloudHubArtifactDeployer.java:85)
        at org.mule.tools.deployment.cloudhub.CloudHubApplicationDeployer.deploy(CloudHubApplicationDeployer.java:41)
        at org.mule.tools.deployment.DefaultDeployer.deploy(DefaultDeployer.java:50)
        at org.mule.tools.maven.mojo.deploy.DeployMojo.doExecute(DeployMojo.java:45)
        at org.mule.tools.maven.mojo.deploy.AbstractMuleDeployerMojo.execute(AbstractMuleDeployerMojo.java:91)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:126)
        at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:328)
        at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:316)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:174)
        at org.apache.maven.lifecycle.internal.MojoExecutor.access$000(MojoExecutor.java:75)
        at org.apache.maven.lifecycle.internal.MojoExecutor$1.run(MojoExecutor.java:162)
        at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute(DefaultMojosExecutionStrategy.java:39)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:159)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:105)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:73)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:53)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:118)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:261)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:173)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:101)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:906)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:283)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:206)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@17.0.10/Native Method)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@17.0.10/NativeMethodAccessorImpl.java:77)
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@17.0.10/DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(java.base@17.0.10/Method.java:568)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:283)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:226)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:407)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:348)
vito-zaccaria commented 2 months ago

Hi @doodo477 , Thanks for spotting this issue. I can see it was fixed in release 4.2.0. Now we can configure the connection provider by setting a java system property -DconnectorProvider=http to enable proxy connections. It accepts three values:

  1. jdk (default value if the property is not set)
  2. apache
  3. http

Regards,