cloudfoundry-community / autosleep

Auto sleep service for CloudFoundry
Apache License 2.0
39 stars 21 forks source link

Missing traces when app enrollment in space fails due to CC_ng exceptions #260

Open gberche-orange opened 7 years ago

gberche-orange commented 7 years ago

We're observing the following errors during app enrollment in spaces.

2016-10-14T16:32:12.64+0200 [APP/0]      OUT ERROR - pool-1-thread-1 - o.c.a.w.SpaceEnroller(106) - cloudfoundry error                                                  
2016-10-14T16:32:12.64+0200 [APP/0]      OUT org.cloudfoundry.autosleep.access.cloudfoundry.CloudFoundryException: org.cloudfoundry.client.v2.CloudFoundryException: CF-NoAppDetectedError(170003): An app was not successfully detected by any available buildpack                                                                             
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.cloudfoundry.autosleep.access.cloudfoundry.CloudFoundryApi.listAliveApplications(CloudFoundryApi.java:370)       
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.cloudfoundry.autosleep.worker.SpaceEnroller.run(SpaceEnroller.java:89)                                           
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)                                              
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at java.util.concurrent.FutureTask.run(FutureTask.java:266)                                                             
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)       
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)                                      
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)                                      
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at java.lang.Thread.run(Thread.java:745)                                                                                
2016-10-14T16:32:12.64+0200 [APP/0]      OUT Caused by: org.cloudfoundry.client.v2.CloudFoundryException: CF-NoAppDetectedError(170003): An app was not successfully detected by any available buildpack
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.cloudfoundry.spring.client.v2.CloudFoundryExceptionBuilder.build(CloudFoundryExceptionBuilder.java:47)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.cloudfoundry.spring.util.AbstractSpringOperations.lambda$null$1(AbstractSpringOperations.java:72)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at reactor.core.publisher.FluxYieldingEmitter$YieldingSignalEmitter.request(FluxYieldingEmitter.java:72)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at reactor.core.util.DeferredSubscription.set(DeferredSubscription.java:71)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at reactor.core.publisher.FluxPublishOn$PublishOnPipeline.onSubscribe(FluxPublishOn.java:126)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at reactor.core.subscriber.SignalEmitter.start(SignalEmitter.java:170)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at reactor.core.publisher.FluxYieldingEmitter.subscribe(FluxYieldingEmitter.java:43)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:152)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at reactor.core.publisher.FluxFlatMap.subscribe(FluxFlatMap.java:169)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at reactor.core.publisher.FluxPublishOn$SourceSubscribeTask.run(FluxPublishOn.java:308)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at reactor.core.publisher.SchedulerGroup$TaskSubscriber.onNext(SchedulerGroup.java:1081)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at reactor.core.publisher.SchedulerGroup$TaskSubscriber.onNext(SchedulerGroup.java:1063)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at reactor.core.publisher.WorkQueueProcessor$QueueSubscriberLoop.run(WorkQueueProcessor.java:842)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    ... 3 common frames omitted
2016-10-14T16:32:12.64+0200 [APP/0]      OUT Caused by: org.springframework.web.client.HttpClientErrorException: 400 Bad Request
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.springframework.security.oauth2.client.http.OAuth2ErrorHandler.handleError(OAuth2ErrorHandler.java:165)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:641)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:597)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.springframework.security.oauth2.client.OAuth2RestTemplate.doExecute(OAuth2RestTemplate.java:128)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:572)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:280)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.cloudfoundry.spring.util.AbstractSpringOperations.lambda$get$3(AbstractSpringOperations.java:85)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    at org.cloudfoundry.spring.util.AbstractSpringOperations.lambda$null$1(AbstractSpringOperations.java:66)
2016-10-14T16:32:12.64+0200 [APP/0]      OUT    ... 14 common frames omitted

/CC @poblin-orange @fbonelle

I'm suspecting the CC_NG improperly fails to list apps in space using the /v2/apps?q=space_guid query and returns an exception that should be return only when listing app instances: https://github.com/cloudfoundry/cloud_controller_ng/blob/173954d8ed2d2b9624d074ba2b277f7bd47c8432/spec/unit/controllers/runtime/instances_controller_spec.rb#L57-L65

The listing of apps should not return this error, we might follow up from https://github.com/cloudfoundry/cloud_controller_ng/blob/173954d8ed2d2b9624d074ba2b277f7bd47c8432/spec/unit/controllers/runtime/apps_controller_spec.rb#L21-L29 and http://apidocs.cloudfoundry.org/245/apps/list_all_apps.html once we obtain HTTP traces to reproduce it.

We're missing some detailed traces from autosleep to reproduce the problem: namely the spaceid for which apps are being listed and potentially the cf-java-client wiretraces.