Parallel build support - Githubissues

mikezerosix commented 3 months ago

We have a lot of proxies and running deploy takes looooong time.

We structured our solution as Maven multi module project. So simply running from root all we can easily deploy them all and we also can easily choose to deploy them individually from each submodule.

But would be great if we would deploy all of them at once using Maven parallel build: mvn -T 10 install (install phase runs the apigee plugin)

We have tried parallel build, but it fail with strange errors like :

[ERROR] Failed to execute goal io.apigee.build-tools.enterprise4g:apigee-edge-maven-plugin:2.5.0:deploy (deploy-bundle) on project shared-request-player: MojoExecutionException: com.google.api.client.http.HttpResponseException: 404 Not Found
[ERROR] {
[ERROR]   "error": {
[ERROR]     "code": 404,
[ERROR]     "message": "generic::not_found: organizations/xxx/sharedflows/shared-xxx/revisions/218 not found",
[ERROR]     "status": "NOT_FOUND",
[ERROR]     "details": [
[ERROR]       {
[ERROR]         "@type": "type.googleapis.com/google.rpc.RequestInfo",
[ERROR]         "requestId": "11261845108213792315"
[ERROR]       }
[ERROR]     ]
[ERROR]   }
[ERROR] }

That shared flow has no dependencies and it deploys without any issues in non-parallel build. Cutting down the half hour deploy time to just 3 minutes being able to run it in parallel would be amazing.

So if possible, please suppport Maven parallel builds.

mikezerosix commented 2 months ago

@ssvaidyanathan Bad news, for some reason the build consistently succeeds in one Apigee org/project, but when deploying to another Apigee org/project, it consistenly fails on the same shared flow trying to get revision 44 when latest rev on that shared flow is 42.
Same familiar error as before:

[ERROR] Failed to execute goal io.apigee.build-tools.enterprise4g:apigee-edge-maven-plugin:2.5.2-rc2:deploy (deploy-bundle) on project shared-response-logging: MojoExecutionException: com.google.api.client.http.HttpResponseException: 404 Not Found
[ERROR] {
[ERROR]   "error": {
[ERROR]     "code": 404,
[ERROR]     "message": "generic::not_found: organizations/xxxx/sharedflows/yyyy/revisions/44 not found",
[ERROR]     "status": "NOT_FOUND",
[ERROR]     "details": [
[ERROR]       {
[ERROR]         "@type": "type.googleapis.com/google.rpc.RequestInfo",
[ERROR]         "requestId": "16420693214369277153"
[ERROR]       }
[ERROR]     ]
[ERROR]   }
[ERROR] }
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal io.apigee.build-tools.enterprise4g:apigee-edge-maven-plugin:2.5.2-rc2:deploy (deploy-bundle) on project shared-response-logging: 
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:375)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:539)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
    at java.lang.Thread.run (Thread.java:840)
Caused by: org.apache.maven.plugin.MojoExecutionException: 
    at io.apigee.buildTools.enterprise4g.mavenplugin.DeployMojo.processHelpfulErrorMessage (DeployMojo.java:334)
    at io.apigee.buildTools.enterprise4g.mavenplugin.DeployMojo.execute (DeployMojo.java:319)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:539)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
    at java.lang.Thread.run (Thread.java:840)
Caused by: java.io.IOException: com.google.api.client.http.HttpResponseException: 404 Not Found
{
  "error": {
    "code": 404,
    "message": "generic::not_found: organizations/xxxx/sharedflows/yyyy/revisions/44 not found",
    "status": "NOT_FOUND",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.RequestInfo",
        "requestId": "16420693214369277153"
      }
    ]
  }
}
    at io.apigee.buildTools.enterprise4g.rest.RestUtil.activateBundleRevision (RestUtil.java:850)
    at io.apigee.buildTools.enterprise4g.rest.RestUtil.activateBundleRevision (RestUtil.java:797)
    at io.apigee.buildTools.enterprise4g.mavenplugin.DeployMojo.doActivateBundle (DeployMojo.java:236)
    at io.apigee.buildTools.enterprise4g.mavenplugin.DeployMojo.execute (DeployMojo.java:299)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:539)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
    at java.lang.Thread.run (Thread.java:840)
Caused by: com.google.api.client.http.HttpResponseException: 404 Not Found
{
  "error": {
    "code": 404,
    "message": "generic::not_found: organizations/xxxx/sharedflows/yyyy/revisions/44 not found",
    "status": "NOT_FOUND",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.RequestInfo",
        "requestId": "16420693214369277153"
      }
    ]
  }
}
    at com.google.api.client.http.HttpRequest.execute (HttpRequest.java:1137)
    at io.apigee.buildTools.enterprise4g.rest.RestUtil.executeAPI (RestUtil.java:924)
    at io.apigee.buildTools.enterprise4g.rest.RestUtil.activateBundleRevision (RestUtil.java:845)
    at io.apigee.buildTools.enterprise4g.rest.RestUtil.activateBundleRevision (RestUtil.java:797)
    at io.apigee.buildTools.enterprise4g.mavenplugin.DeployMojo.doActivateBundle (DeployMojo.java:236)
    at io.apigee.buildTools.enterprise4g.mavenplugin.DeployMojo.execute (DeployMojo.java:299)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:539)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
    at java.lang.Thread.run (Thread.java:840)
[ERROR] Failed to execute goal com.apigee.edge.config:apigee-config-maven-plugin:2.8.1:keyvaluemaps (create-config-keyvaluemaps) on project config: null: MojoExecutionException -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.apigee.edge.config:apigee-config-maven-plugin:2.8.1:keyvaluemaps (create-config-keyvaluemaps) on project config: null
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:375)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:539)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
    at java.lang.Thread.run (Thread.java:840)
Caused by: org.apache.maven.plugin.MojoExecutionException
    at com.apigee.edge.config.mavenplugin.GatewayAbstractMojo.getOrgConfig (GatewayAbstractMojo.java:638)
    at com.apigee.edge.config.mavenplugin.KVMMojo.execute (KVMMojo.java:378)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:539)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
    at java.lang.Thread.run (Thread.java:840)

ssvaidyanathan commented 2 months ago

@mikezerosix - Can you try just deploying that sharedflow using the plugin (comment out other modules)

Also - can the sharedflows be executed in 'async' mode?

mikezerosix commented 2 months ago

@ssvaidyanathan I tried running sharedflows only and that did not cause errors. There is only 7, so that is not definiteve proof the flows work alone. As there seem to be a lot of randomness on the run. Shared flows can not be ran in async.

ssvaidyanathan commented 2 months ago

@mikezerosix - sorry for the delay. I was out on a personal emergency.. Are we still seeing issues? And is this only with sharedflows?

JanneHoltta commented 1 month ago

Hi, @mikezerosix is no longer working on the project so I will continue the conversation (Micheal if you are still there you are very welcome to answer as well :D). So yes we are still seeing the same issues with sharedflows

ssvaidyanathan commented 4 weeks ago

@JanneHoltta - sorry for the delayed response. I have been out of office. Can you please confirm if this is happening just for sharedflows and NOT for proxies?

1) Are you testing proxies or sharedflow separately?? Or do you have them all mixed as submodules and thats failing? If you are mixing them, can you try them individually and see if its causing any issue? 2) In the logs, are you seeing the wrong revision for the sharedflow thats failing?

The code for proxies and sharedflow is the same. Not sure why its only failing for sharedflows. Appreciate if you can response the questions above so that I can take a look

ssvaidyanathan commented 3 weeks ago

@JanneHoltta - looks like its a known issue

For now, I dont think its plugin issue. The best way is to probably reduce the number of submodules you have in your pom and split it into multiple runs.

apigee / apigee-deploy-maven-plugin

Parallel build support #233