search

cloudfoundry / cf-java-client

Java Client Library for Cloud Foundry
Apache License 2.0
328 stars 318 forks source link

Frequent 500 Error while getting staging logs. #241

Closed joek closed 9 years ago

joek commented 9 years ago

Hi, we are experiencing frequently failed deployments because of 500 Errors using the cf-java-client. 

build   24-Nov-2014 13:08:35    Download http://repo1.maven.org/maven2/org/cloudfoundry/cf-gradle-plugin/1.0.5/cf-gradle-plugin-1.0.5.pom
build   24-Nov-2014 13:08:36    Download http://repo1.maven.org/maven2/org/cloudfoundry/cloudfoundry-client-lib/1.0.5/cloudfoundry-client-lib-1.0.5.pom
build   24-Nov-2014 13:08:36    Download http://repo1.maven.org/maven2/org/apache/tomcat/embed/tomcat-embed-websocket/8.0.15/tomcat-embed-websocket-8.0.15.pom
build   24-Nov-2014 13:08:36    Download http://repo1.maven.org/maven2/com/google/protobuf/protobuf-java/2.6.1/protobuf-java-2.6.1.pom
build   24-Nov-2014 13:08:36    Download http://repo1.maven.org/maven2/org/apache/tomcat/embed/tomcat-embed-core/8.0.15/tomcat-embed-core-8.0.15.pom
build   24-Nov-2014 13:08:37    Download http://repo1.maven.org/maven2/org/cloudfoundry/cf-gradle-plugin/1.0.5/cf-gradle-plugin-1.0.5.jar
build   24-Nov-2014 13:08:37    Download http://repo1.maven.org/maven2/org/cloudfoundry/cloudfoundry-client-lib/1.0.5/cloudfoundry-client-lib-1.0.5.jar
build   24-Nov-2014 13:08:37    Download http://repo1.maven.org/maven2/org/apache/tomcat/embed/tomcat-embed-websocket/8.0.15/tomcat-embed-websocket-8.0.15.jar
build   24-Nov-2014 13:08:37    Download http://repo1.maven.org/maven2/com/google/protobuf/protobuf-java/2.6.1/protobuf-java-2.6.1.jar
build   24-Nov-2014 13:08:37    Download http://repo1.maven.org/maven2/org/apache/tomcat/embed/tomcat-embed-core/8.0.15/tomcat-embed-core-8.0.15.jar
build   24-Nov-2014 13:08:47    :cfDeploy
build   24-Nov-2014 13:08:47    Currently active variants are [devportal-blue], deploying devportal-green
build   24-Nov-2014 13:08:48    Updating application devportal-green
build   24-Nov-2014 13:08:52    Uploading file target/devportal-crowd-integration-1.0-SNAPSHOT.war
build   24-Nov-2014 13:09:30    Starting devportal-green
build   24-Nov-2014 13:09:39    GET request for "http://33ba8a1b2767409faff19a06d2754221.cf1.hybris.com/staging_tasks/9ebc1cde87144b0eaff519060c2b71d8/file_path?hmac=83d3d607f0d6b7fad213069e54a1a7192fff63b29e94cdf7978c9e8ac4def64323a00e4e91574ed86747d944419fe4e5859c45fe8bff2a81544a8608dac6736a&path=/tmp/staged/logs/staging_task.log&timestamp=1416830976&tail&tail_offset=0" resulted in 500 (Internal Server Error); invoking error handler
build   24-Nov-2014 13:09:39    :cfDeploy FAILED
error   24-Nov-2014 13:09:39    
error   24-Nov-2014 13:09:39    FAILURE: Build failed with an exception.
error   24-Nov-2014 13:09:39    
error   24-Nov-2014 13:09:39    * What went wrong:
error   24-Nov-2014 13:09:39    Execution failed for task ':cfDeploy'.
error   24-Nov-2014 13:09:39    > 500 Internal Server Error
error   24-Nov-2014 13:09:39    
error   24-Nov-2014 13:09:39    * Try:
error   24-Nov-2014 13:09:39    Run with --info or --debug option to get more log output.
error   24-Nov-2014 13:09:39    
error   24-Nov-2014 13:09:39    * Exception is:
error   24-Nov-2014 13:09:39    org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':cfDeploy'.
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:296)
error   24-Nov-2014 13:09:39        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
error   24-Nov-2014 13:09:39        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
error   24-Nov-2014 13:09:39        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
error   24-Nov-2014 13:09:39        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
error   24-Nov-2014 13:09:39        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:86)
error   24-Nov-2014 13:09:39        at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
error   24-Nov-2014 13:09:39        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)
error   24-Nov-2014 13:09:39        at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
error   24-Nov-2014 13:09:39        at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:67)
error   24-Nov-2014 13:09:39        at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
error   24-Nov-2014 13:09:39        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)
error   24-Nov-2014 13:09:39        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:54)
error   24-Nov-2014 13:09:39        at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:148)
error   24-Nov-2014 13:09:39        at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:105)
error   24-Nov-2014 13:09:39        at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:85)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:81)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:39)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:29)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:50)
error   24-Nov-2014 13:09:39        at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:171)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.Main.doAction(Main.java:33)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
error   24-Nov-2014 13:09:39        at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
error   24-Nov-2014 13:09:39        at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
error   24-Nov-2014 13:09:39        at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:127)
error   24-Nov-2014 13:09:39        at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:55)
error   24-Nov-2014 13:09:39    Caused by: org.springframework.web.client.HttpServerErrorException: 500 Internal Server Error
error   24-Nov-2014 13:09:39        at org.cloudfoundry.client.lib.rest.CloudControllerResponseErrorHandler.handleError(CloudControllerResponseErrorHandler.java:26)
error   24-Nov-2014 13:09:39        at org.springframework.web.client.RestTemplate.handleResponseError(RestTemplate.java:557)
error   24-Nov-2014 13:09:39        at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:515)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.client.lib.rest.LoggingRestTemplate.doExecute(LoggingRestTemplate.java:54)
error   24-Nov-2014 13:09:39        at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:480)
error   24-Nov-2014 13:09:39        at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:245)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.client.lib.rest.CloudControllerClientImpl.getStagingLogs(CloudControllerClientImpl.java:317)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.client.lib.CloudFoundryClient.getStagingLogs(CloudFoundryClient.java:358)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.client.lib.CloudFoundryOperations$getStagingLogs$10.call(Unknown Source)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.gradle.tasks.StartCloudFoundryHelper.showStagingStatus(StartCloudFoundryHelper.groovy:32)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.gradle.tasks.StartCloudFoundryHelper.invokeMethod(StartCloudFoundryHelper.groovy)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.gradle.tasks.StartCloudFoundryHelper.showAppStartup(StartCloudFoundryHelper.groovy:22)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.gradle.tasks.StartCloudFoundryHelper.invokeMethod(StartCloudFoundryHelper.groovy)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.gradle.tasks.StartCloudFoundryHelper.startApplication(StartCloudFoundryHelper.groovy:16)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.gradle.tasks.DeployCloudFoundryTask$_deploy_closure1.doCall(DeployCloudFoundryTask.groovy:63)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.gradle.tasks.AbstractCloudFoundryTask.withCloudFoundryClient(AbstractCloudFoundryTask.groovy:64)
error   24-Nov-2014 13:09:39        at org.cloudfoundry.gradle.tasks.DeployCloudFoundryTask.deploy(DeployCloudFoundryTask.groovy:35)
error   24-Nov-2014 13:09:39        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:218)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:211)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:200)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:570)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:553)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
error   24-Nov-2014 13:09:39        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
error   24-Nov-2014 13:09:39        ... 47 more
error   24-Nov-2014 13:09:39    
build   24-Nov-2014 13:09:39    
build   24-Nov-2014 13:09:39    BUILD FAILED
build   24-Nov-2014 13:09:39    
build   24-Nov-2014 13:09:39    Total time: 1 mins 10.808 secs
´´´

The Issue is not related to gradle, this is happen in maven as well. Do you know an workaround to avoid errors like this?
cf-gitbot commented 9 years ago

We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: https://www.pivotaltracker.com/story/show/83304022.

scottfrederick commented 9 years ago

This is a duplicate of #222. The previous issue was most often reported when using the Gradle plugin, but it affected all users of the CF Java client (including the Gradle, Maven, and Eclipse plugins).

The issue was with the staging_tasks endpoint on Cloud Controller, due to a downstream issue in DEA. See https://www.pivotaltracker.com/story/show/80319590 for more details. You might need to apply this change to your cf-release.

If have questions or need assistance applying the DEA change, please post to the vcap-dev mailing list so the Pivotal CF Runtime team will see it.

scottfrederick commented 9 years ago

@joek I'm going to close this pending your confirmation that it is fixed by your updating your cf-release. Please re-open it if you can confirm that it is different from the DEA issue linked above.