search

spring-cloud / spring-cloud-netflix

Integration with Netflix OSS components
http://cloud.spring.io/spring-cloud-netflix/
Apache License 2.0
4.88k stars 2.44k forks source link

Spring cloud turbine.stream reports no data #820

Closed tianhongbo closed 8 years ago

tianhongbo commented 8 years ago

I have deployed the turbine service together with a cluster of "DEVICE-SERVICE" in the cloud. It works well at beginning (The Hystrix data can be viewed both at Hystix Dashboard and terminal). However, it broken after about one day running. I reboot turbine service, but it occurred again.

POM.xml

    <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-starter-turbine</artifactId>
      <version>1.0.1.RELEASE</version>
    </dependency>

Code:

    import org.springframework.boot.SpringApplication;
    import org.springframework.boot.autoconfigure.SpringBootApplication;
    import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
    import org.springframework.cloud.netflix.turbine.EnableTurbine;
    import springfox.documentation.swagger2.annotations.EnableSwagger2;

    @EnableDiscoveryClient

    @EnableTurbine

    @SpringBootApplication

    @EnableSwagger2

    public class Application {

        public static void main(String[] args) {
            SpringApplication.run(Application.class, args);
        }
    }

BTW: When turbine service works well, the data can be viewed with the following:

  1. On the Dashboard, the data can be viewed via "http://turbine-service.mi-ae.cn/turbine.stream?cluster=DEVICE-SERVICE"
  2. On the terminal, "CURL http://turbine-service.mi-ae.cn/turbine.stream?cluster=DEVICE-SERVICE"

When turbine service breaks, it looks like this:

  1. On the Dashboard: Click to see the picture
  2. On the terminal: Click to see the picture

Hope everyone can help!

dsyer commented 8 years ago

The server is still sending data, but not actual metrics. Are you sure there are some hystrix commands out in your cluster sending their own metrics? Maybe there is just no data?

tianhongbo commented 8 years ago

Yes, I have double checked and confirmed that there are some hysteria commands in the cluster sending their own metrics. I checked in tow ways: 1) curl http://device-service.mi-ae.cn/hystrix.stream 2) use Hystrix Dashboard with url of "http://device-service.mi-ae.cn/hystrix.stream". Both work well with the metrics date output.

By the way, I did more experiments with local setup. Beside keeping the same setup in the cloud, I run the same turbine server in my local MacPro. The local turbine works very well, and It can fetch all the metrics data from "DEVICE-SERVICE" in the cloud. However, the turbine server running in the cloud does not.

I did a quick comparison of logs output from these two turbine servers, and found three strange information here.

  1. Local turbine server uses nio(no block io), but cloud turbine server uses io(block io)
  2. local turbine server less than 10 [nio-8989-exec-9] // 8989 is turbine port. However, cloud turbine server has maximum 24 [io-62188-exec-24] // 62188 is turbine port.
  3. In the console output, I found [io-62188-exec-x] will change to [o-62188-exec-xx] when the number after 'exec-' is above 9. I am not sure if it's correct or bug.

Here is some log from both turbine servers: From turbine running at local MacPro:

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:103)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:65)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
    at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:124)
    at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
    at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:172)
    at org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:139)
    at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:197)
    at org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:41)
    at org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:320)
    at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:116)
    at org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:256)
    at org.apache.coyote.Response.doWrite(Response.java:501)
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:388)
    ... 52 common frames omitted

2016-02-12 14:38:04.153  INFO 55976 --- [nio-8989-exec-3] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_b2f939a4-1a95-4708-a2c2-61263f4efd4d
2016-02-12 14:38:04.153  INFO 55976 --- [nio-8989-exec-3] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_b2f939a4-1a95-4708-a2c2-61263f4efd4d
2016-02-12 14:38:04.153  INFO 55976 --- [ool-28-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:04.206  INFO 55976 --- [nio-8989-exec-3] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_b2f939a4-1a95-4708-a2c2-61263f4efd4d tuple running: false
2016-02-12 14:38:04.206  INFO 55976 --- [nio-8989-exec-3] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StreamingHandler_b3bedc34-5df4-44f0-82ae-bb11a350cc0a, StaticListener_For_Aggregator]
2016-02-12 14:38:04.206  INFO 55976 --- [nio-8989-exec-3] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@2af0264f]
2016-02-12 14:38:12.981  INFO 55976 --- [nio-8989-exec-6] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:12.981  INFO 55976 --- [nio-8989-exec-6] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:12.982  INFO 55976 --- [nio-8989-exec-6] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:12.982  INFO 55976 --- [nio-8989-exec-6] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:12.982  INFO 55976 --- [nio-8989-exec-6] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_8c45789d-ea39-4b54-8c94-4882d447a4e5
2016-02-12 14:38:12.982  INFO 55976 --- [nio-8989-exec-6] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_8c45789d-ea39-4b54-8c94-4882d447a4e5
2016-02-12 14:38:12.982  INFO 55976 --- [nio-8989-exec-6] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_8c45789d-ea39-4b54-8c94-4882d447a4e5
2016-02-12 14:38:12.982  INFO 55976 --- [nio-8989-exec-6] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_b3bedc34-5df4-44f0-82ae-bb11a350cc0a, StaticListener_For_Aggregator, StreamingHandler_8c45789d-ea39-4b54-8c94-4882d447a4e5]
2016-02-12 14:38:12.982  INFO 55976 --- [nio-8989-exec-6] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:38:14.603 ERROR 55976 --- [nio-8989-exec-1] c.n.t.s.TurbineStreamingConnection       : Got exception when writing to response stream

org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:393)
    at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:426)
    at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:342)
    at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:317)
    at org.apache.catalina.connector.Response.flushBuffer(Response.java:511)
    at org.apache.catalina.connector.ResponseFacade.flushBuffer(ResponseFacade.java:318)
    at com.netflix.turbine.streaming.servlet.SynchronizedHttpServletResponse.flushBuffer(SynchronizedHttpServletResponse.java:79)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet$ServletStreamHandler.noData(TurbineStreamServlet.java:226)
    at com.netflix.turbine.streaming.TurbineStreamingConnection.waitOnConnection(TurbineStreamingConnection.java:214)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.streamFromCluster(TurbineStreamServlet.java:136)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.doGet(TurbineStreamServlet.java:98)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:300)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:103)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:65)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
    at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:124)
    at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
    at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:172)
    at org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:139)
    at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:197)
    at org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:41)
    at org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:320)
    at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:116)
    at org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:256)
    at org.apache.coyote.Response.doWrite(Response.java:501)
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:388)
    ... 52 common frames omitted

2016-02-12 14:38:15.413  INFO 55976 --- [nio-8989-exec-7] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:15.413  INFO 55976 --- [nio-8989-exec-7] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:15.413  INFO 55976 --- [nio-8989-exec-7] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:15.413  INFO 55976 --- [nio-8989-exec-7] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:15.413  INFO 55976 --- [nio-8989-exec-7] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_7b280414-d7dd-40bd-acd1-284fce3e03ff
2016-02-12 14:38:15.414  INFO 55976 --- [nio-8989-exec-7] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_7b280414-d7dd-40bd-acd1-284fce3e03ff
2016-02-12 14:38:15.414  INFO 55976 --- [nio-8989-exec-7] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_7b280414-d7dd-40bd-acd1-284fce3e03ff
2016-02-12 14:38:15.414  INFO 55976 --- [nio-8989-exec-7] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_7b280414-d7dd-40bd-acd1-284fce3e03ff, StreamingHandler_b3bedc34-5df4-44f0-82ae-bb11a350cc0a, StaticListener_For_Aggregator, StreamingHandler_8c45789d-ea39-4b54-8c94-4882d447a4e5]
2016-02-12 14:38:15.414  INFO 55976 --- [nio-8989-exec-7] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:38:15.986  INFO 55976 --- [nio-8989-exec-6] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_8c45789d-ea39-4b54-8c94-4882d447a4e5
2016-02-12 14:38:15.986  INFO 55976 --- [nio-8989-exec-6] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_8c45789d-ea39-4b54-8c94-4882d447a4e5
2016-02-12 14:38:15.986  INFO 55976 --- [ool-30-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:16.039  INFO 55976 --- [nio-8989-exec-6] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_8c45789d-ea39-4b54-8c94-4882d447a4e5 tuple running: false
2016-02-12 14:38:16.040  INFO 55976 --- [nio-8989-exec-6] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StreamingHandler_7b280414-d7dd-40bd-acd1-284fce3e03ff, StreamingHandler_b3bedc34-5df4-44f0-82ae-bb11a350cc0a, StaticListener_For_Aggregator]
2016-02-12 14:38:16.040  INFO 55976 --- [nio-8989-exec-6] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@139e366]
2016-02-12 14:38:17.604  INFO 55976 --- [nio-8989-exec-1] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_b3bedc34-5df4-44f0-82ae-bb11a350cc0a
2016-02-12 14:38:17.604  INFO 55976 --- [nio-8989-exec-1] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_b3bedc34-5df4-44f0-82ae-bb11a350cc0a
2016-02-12 14:38:17.604  INFO 55976 --- [ool-29-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:17.657  INFO 55976 --- [nio-8989-exec-1] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_b3bedc34-5df4-44f0-82ae-bb11a350cc0a tuple running: false
2016-02-12 14:38:17.657  INFO 55976 --- [nio-8989-exec-1] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StreamingHandler_7b280414-d7dd-40bd-acd1-284fce3e03ff, StaticListener_For_Aggregator]
2016-02-12 14:38:17.657  INFO 55976 --- [nio-8989-exec-1] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@124c898]
2016-02-12 14:38:17.681  INFO 55976 --- [nio-8989-exec-8] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:17.681  INFO 55976 --- [nio-8989-exec-8] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:17.681  INFO 55976 --- [nio-8989-exec-8] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:17.681  INFO 55976 --- [nio-8989-exec-8] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:17.681  INFO 55976 --- [nio-8989-exec-8] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_2948f977-8e72-40a2-bf30-770e53fba30f
2016-02-12 14:38:17.681  INFO 55976 --- [nio-8989-exec-8] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_2948f977-8e72-40a2-bf30-770e53fba30f
2016-02-12 14:38:17.681  INFO 55976 --- [nio-8989-exec-8] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_2948f977-8e72-40a2-bf30-770e53fba30f
2016-02-12 14:38:17.682  INFO 55976 --- [nio-8989-exec-8] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_7b280414-d7dd-40bd-acd1-284fce3e03ff, StaticListener_For_Aggregator, StreamingHandler_2948f977-8e72-40a2-bf30-770e53fba30f]
2016-02-12 14:38:17.682  INFO 55976 --- [nio-8989-exec-8] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:38:20.757  INFO 55976 --- [nio-8989-exec-9] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:20.757  INFO 55976 --- [nio-8989-exec-9] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:20.757  INFO 55976 --- [nio-8989-exec-9] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:20.757  INFO 55976 --- [nio-8989-exec-9] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:20.757  INFO 55976 --- [nio-8989-exec-9] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_f3abb14e-5040-44b8-ac27-8bf04d648667
2016-02-12 14:38:20.757  INFO 55976 --- [nio-8989-exec-9] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_f3abb14e-5040-44b8-ac27-8bf04d648667
2016-02-12 14:38:20.757  INFO 55976 --- [nio-8989-exec-9] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_f3abb14e-5040-44b8-ac27-8bf04d648667
2016-02-12 14:38:20.757  INFO 55976 --- [nio-8989-exec-9] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_7b280414-d7dd-40bd-acd1-284fce3e03ff, StaticListener_For_Aggregator, StreamingHandler_2948f977-8e72-40a2-bf30-770e53fba30f, StreamingHandler_f3abb14e-5040-44b8-ac27-8bf04d648667]
2016-02-12 14:38:20.757  INFO 55976 --- [nio-8989-exec-9] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:38:21.416  INFO 55976 --- [nio-8989-exec-7] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_7b280414-d7dd-40bd-acd1-284fce3e03ff
2016-02-12 14:38:21.418  INFO 55976 --- [nio-8989-exec-7] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_7b280414-d7dd-40bd-acd1-284fce3e03ff
2016-02-12 14:38:21.419  INFO 55976 --- [ool-31-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:21.472  INFO 55976 --- [nio-8989-exec-7] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_7b280414-d7dd-40bd-acd1-284fce3e03ff tuple running: false
2016-02-12 14:38:21.472  INFO 55976 --- [nio-8989-exec-7] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator, StreamingHandler_2948f977-8e72-40a2-bf30-770e53fba30f, StreamingHandler_f3abb14e-5040-44b8-ac27-8bf04d648667]
2016-02-12 14:38:21.472  INFO 55976 --- [nio-8989-exec-7] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@2f88724b]
2016-02-12 14:38:23.691  INFO 55976 --- [nio-8989-exec-8] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_2948f977-8e72-40a2-bf30-770e53fba30f
2016-02-12 14:38:23.691  INFO 55976 --- [nio-8989-exec-8] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_2948f977-8e72-40a2-bf30-770e53fba30f
2016-02-12 14:38:23.691  INFO 55976 --- [ool-32-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:23.743  INFO 55976 --- [nio-8989-exec-8] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_2948f977-8e72-40a2-bf30-770e53fba30f tuple running: false
2016-02-12 14:38:23.743  INFO 55976 --- [nio-8989-exec-8] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator, StreamingHandler_f3abb14e-5040-44b8-ac27-8bf04d648667]
2016-02-12 14:38:23.743  INFO 55976 --- [nio-8989-exec-8] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@11003ca]
2016-02-12 14:38:26.767  INFO 55976 --- [nio-8989-exec-9] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_f3abb14e-5040-44b8-ac27-8bf04d648667
2016-02-12 14:38:26.767  INFO 55976 --- [nio-8989-exec-9] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_f3abb14e-5040-44b8-ac27-8bf04d648667
2016-02-12 14:38:26.767  INFO 55976 --- [ool-33-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:26.821  INFO 55976 --- [nio-8989-exec-9] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_f3abb14e-5040-44b8-ac27-8bf04d648667 tuple running: false
2016-02-12 14:38:26.821  INFO 55976 --- [nio-8989-exec-9] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator]
2016-02-12 14:38:26.821  INFO 55976 --- [nio-8989-exec-9] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@3e78e640]
2016-02-12 14:38:27.757  INFO 55976 --- [nio-8989-exec-4] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:27.757  INFO 55976 --- [nio-8989-exec-4] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:27.758  INFO 55976 --- [nio-8989-exec-4] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:27.758  INFO 55976 --- [nio-8989-exec-4] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:27.758  INFO 55976 --- [nio-8989-exec-4] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_e4a35040-9fac-443f-b224-dcb496d17d77
2016-02-12 14:38:27.758  INFO 55976 --- [nio-8989-exec-4] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_e4a35040-9fac-443f-b224-dcb496d17d77
2016-02-12 14:38:27.758  INFO 55976 --- [nio-8989-exec-4] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_e4a35040-9fac-443f-b224-dcb496d17d77
2016-02-12 14:38:27.758  INFO 55976 --- [nio-8989-exec-4] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_e4a35040-9fac-443f-b224-dcb496d17d77, StaticListener_For_Aggregator]
2016-02-12 14:38:27.758  INFO 55976 --- [nio-8989-exec-4] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:38:31.492  INFO 55976 --- [nio-8989-exec-5] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:31.492  INFO 55976 --- [nio-8989-exec-5] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:31.492  INFO 55976 --- [nio-8989-exec-5] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:31.492  INFO 55976 --- [nio-8989-exec-5] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:31.492  INFO 55976 --- [nio-8989-exec-5] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_be69a683-878b-43d4-bdcb-8fb7af8a857e
2016-02-12 14:38:31.492  INFO 55976 --- [nio-8989-exec-5] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_be69a683-878b-43d4-bdcb-8fb7af8a857e
2016-02-12 14:38:31.492  INFO 55976 --- [nio-8989-exec-5] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_be69a683-878b-43d4-bdcb-8fb7af8a857e
2016-02-12 14:38:31.492  INFO 55976 --- [nio-8989-exec-5] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_be69a683-878b-43d4-bdcb-8fb7af8a857e, StreamingHandler_e4a35040-9fac-443f-b224-dcb496d17d77, StaticListener_For_Aggregator]
2016-02-12 14:38:31.492  INFO 55976 --- [nio-8989-exec-5] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:38:33.766  INFO 55976 --- [nio-8989-exec-4] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_e4a35040-9fac-443f-b224-dcb496d17d77
2016-02-12 14:38:33.766  INFO 55976 --- [nio-8989-exec-4] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_e4a35040-9fac-443f-b224-dcb496d17d77
2016-02-12 14:38:33.766  INFO 55976 --- [ool-34-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:33.821  INFO 55976 --- [nio-8989-exec-4] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_e4a35040-9fac-443f-b224-dcb496d17d77 tuple running: false
2016-02-12 14:38:33.821  INFO 55976 --- [nio-8989-exec-4] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StreamingHandler_be69a683-878b-43d4-bdcb-8fb7af8a857e, StaticListener_For_Aggregator]
2016-02-12 14:38:33.821  INFO 55976 --- [nio-8989-exec-4] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@ef1051e]
2016-02-12 14:38:34.316  INFO 55976 --- [nio-8989-exec-2] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:34.316  INFO 55976 --- [nio-8989-exec-2] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:34.316  INFO 55976 --- [nio-8989-exec-2] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:34.316  INFO 55976 --- [nio-8989-exec-2] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:34.316  INFO 55976 --- [nio-8989-exec-2] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_d064fbe8-a9d8-4c20-8490-5bd74fd485f0
2016-02-12 14:38:34.316  INFO 55976 --- [nio-8989-exec-2] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_d064fbe8-a9d8-4c20-8490-5bd74fd485f0
2016-02-12 14:38:34.316  INFO 55976 --- [nio-8989-exec-2] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_d064fbe8-a9d8-4c20-8490-5bd74fd485f0
2016-02-12 14:38:34.316  INFO 55976 --- [nio-8989-exec-2] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_be69a683-878b-43d4-bdcb-8fb7af8a857e, StaticListener_For_Aggregator, StreamingHandler_d064fbe8-a9d8-4c20-8490-5bd74fd485f0]
2016-02-12 14:38:34.316  INFO 55976 --- [nio-8989-exec-2] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:38:34.497 ERROR 55976 --- [nio-8989-exec-5] c.n.t.s.TurbineStreamingConnection       : Got exception when writing to response stream

org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:393)
    at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:426)
    at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:342)
    at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:317)
    at org.apache.catalina.connector.Response.flushBuffer(Response.java:511)
    at org.apache.catalina.connector.ResponseFacade.flushBuffer(ResponseFacade.java:318)
    at com.netflix.turbine.streaming.servlet.SynchronizedHttpServletResponse.flushBuffer(SynchronizedHttpServletResponse.java:79)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet$ServletStreamHandler.noData(TurbineStreamServlet.java:226)
    at com.netflix.turbine.streaming.TurbineStreamingConnection.waitOnConnection(TurbineStreamingConnection.java:214)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.streamFromCluster(TurbineStreamServlet.java:136)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.doGet(TurbineStreamServlet.java:98)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:300)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:103)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:65)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
    at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:124)
    at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
    at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:172)
    at org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:139)
    at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:197)
    at org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:41)
    at org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:320)
    at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:116)
    at org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:256)
    at org.apache.coyote.Response.doWrite(Response.java:501)
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:388)
    ... 52 common frames omitted

2016-02-12 14:38:36.345  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
2016-02-12 14:38:36.346  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: AUTH-SERVICE
2016-02-12 14:38:36.346  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: AUTH-SERVICE, size=2
2016-02-12 14:38:36.347  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: DEVICE-SERVICE
2016-02-12 14:38:36.347  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: DEVICE-SERVICE, size=2
2016-02-12 14:38:36.348  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Retrieved hosts from InstanceDiscovery: 4
2016-02-12 14:38:36.348  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Found hosts that have been previously terminated: 0
2016-02-12 14:38:36.348  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Hosts up:2, hosts down: 0
2016-02-12 14:38:36.940  INFO 55976 --- [io-8989-exec-10] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:36.941  INFO 55976 --- [io-8989-exec-10] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:36.941  INFO 55976 --- [io-8989-exec-10] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:36.941  INFO 55976 --- [io-8989-exec-10] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:36.941  INFO 55976 --- [io-8989-exec-10] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8
2016-02-12 14:38:36.941  INFO 55976 --- [io-8989-exec-10] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8
2016-02-12 14:38:36.941  INFO 55976 --- [io-8989-exec-10] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8
2016-02-12 14:38:36.941  INFO 55976 --- [io-8989-exec-10] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_be69a683-878b-43d4-bdcb-8fb7af8a857e, StaticListener_For_Aggregator, StreamingHandler_d064fbe8-a9d8-4c20-8490-5bd74fd485f0, StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8]
2016-02-12 14:38:36.941  INFO 55976 --- [io-8989-exec-10] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:38:37.321 ERROR 55976 --- [nio-8989-exec-2] c.n.t.s.TurbineStreamingConnection       : Got exception when writing to response stream

org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:393)
    at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:426)
    at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:342)
    at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:317)
    at org.apache.catalina.connector.Response.flushBuffer(Response.java:511)
    at org.apache.catalina.connector.ResponseFacade.flushBuffer(ResponseFacade.java:318)
    at com.netflix.turbine.streaming.servlet.SynchronizedHttpServletResponse.flushBuffer(SynchronizedHttpServletResponse.java:79)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet$ServletStreamHandler.noData(TurbineStreamServlet.java:226)
    at com.netflix.turbine.streaming.TurbineStreamingConnection.waitOnConnection(TurbineStreamingConnection.java:214)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.streamFromCluster(TurbineStreamServlet.java:136)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.doGet(TurbineStreamServlet.java:98)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:300)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:103)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:65)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
    at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:124)
    at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
    at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:172)
    at org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:139)
    at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:197)
    at org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:41)
    at org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:320)
    at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:116)
    at org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:256)
    at org.apache.coyote.Response.doWrite(Response.java:501)
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:388)
    ... 52 common frames omitted

2016-02-12 14:38:37.501  INFO 55976 --- [nio-8989-exec-5] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_be69a683-878b-43d4-bdcb-8fb7af8a857e
2016-02-12 14:38:37.501  INFO 55976 --- [nio-8989-exec-5] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_be69a683-878b-43d4-bdcb-8fb7af8a857e
2016-02-12 14:38:37.501  INFO 55976 --- [ool-35-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:37.555  INFO 55976 --- [nio-8989-exec-5] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_be69a683-878b-43d4-bdcb-8fb7af8a857e tuple running: false
2016-02-12 14:38:37.555  INFO 55976 --- [nio-8989-exec-5] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator, StreamingHandler_d064fbe8-a9d8-4c20-8490-5bd74fd485f0, StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8]
2016-02-12 14:38:37.555  INFO 55976 --- [nio-8989-exec-5] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@6dd4be13]
2016-02-12 14:38:39.147  INFO 55976 --- [nio-8989-exec-3] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:39.148  INFO 55976 --- [nio-8989-exec-3] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:39.148  INFO 55976 --- [nio-8989-exec-3] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:39.148  INFO 55976 --- [nio-8989-exec-3] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:39.148  INFO 55976 --- [nio-8989-exec-3] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_b49d738a-00ed-479e-b131-38878d04897f
2016-02-12 14:38:39.148  INFO 55976 --- [nio-8989-exec-3] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_b49d738a-00ed-479e-b131-38878d04897f
2016-02-12 14:38:39.148  INFO 55976 --- [nio-8989-exec-3] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_b49d738a-00ed-479e-b131-38878d04897f
2016-02-12 14:38:39.148  INFO 55976 --- [nio-8989-exec-3] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StaticListener_For_Aggregator, StreamingHandler_d064fbe8-a9d8-4c20-8490-5bd74fd485f0, StreamingHandler_b49d738a-00ed-479e-b131-38878d04897f, StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8]
2016-02-12 14:38:39.148  INFO 55976 --- [nio-8989-exec-3] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:38:40.322  INFO 55976 --- [nio-8989-exec-2] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_d064fbe8-a9d8-4c20-8490-5bd74fd485f0
2016-02-12 14:38:40.322  INFO 55976 --- [nio-8989-exec-2] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_d064fbe8-a9d8-4c20-8490-5bd74fd485f0
2016-02-12 14:38:40.322  INFO 55976 --- [ool-36-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:40.377  INFO 55976 --- [nio-8989-exec-2] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_d064fbe8-a9d8-4c20-8490-5bd74fd485f0 tuple running: false
2016-02-12 14:38:40.377  INFO 55976 --- [nio-8989-exec-2] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator, StreamingHandler_b49d738a-00ed-479e-b131-38878d04897f, StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8]
2016-02-12 14:38:40.377  INFO 55976 --- [nio-8989-exec-2] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@1da923ff]
2016-02-12 14:38:40.931  INFO 55976 --- [nio-8989-exec-6] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:40.931  INFO 55976 --- [nio-8989-exec-6] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:40.931  INFO 55976 --- [nio-8989-exec-6] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:40.931  INFO 55976 --- [nio-8989-exec-6] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:40.931  INFO 55976 --- [nio-8989-exec-6] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_315f98ad-26db-4ee7-8330-486caab147e9
2016-02-12 14:38:40.931  INFO 55976 --- [nio-8989-exec-6] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_315f98ad-26db-4ee7-8330-486caab147e9
2016-02-12 14:38:40.931  INFO 55976 --- [nio-8989-exec-6] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_315f98ad-26db-4ee7-8330-486caab147e9
2016-02-12 14:38:40.932  INFO 55976 --- [nio-8989-exec-6] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_315f98ad-26db-4ee7-8330-486caab147e9, StaticListener_For_Aggregator, StreamingHandler_b49d738a-00ed-479e-b131-38878d04897f, StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8]
2016-02-12 14:38:40.932  INFO 55976 --- [nio-8989-exec-6] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:38:42.151  INFO 55976 --- [nio-8989-exec-3] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_b49d738a-00ed-479e-b131-38878d04897f
2016-02-12 14:38:42.151  INFO 55976 --- [nio-8989-exec-3] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_b49d738a-00ed-479e-b131-38878d04897f
2016-02-12 14:38:42.151  INFO 55976 --- [ool-38-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:42.206  INFO 55976 --- [nio-8989-exec-3] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_b49d738a-00ed-479e-b131-38878d04897f tuple running: false
2016-02-12 14:38:42.206  INFO 55976 --- [nio-8989-exec-3] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StreamingHandler_315f98ad-26db-4ee7-8330-486caab147e9, StaticListener_For_Aggregator, StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8]
2016-02-12 14:38:42.206  INFO 55976 --- [nio-8989-exec-3] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@7b0e04c8]
2016-02-12 14:38:42.948  INFO 55976 --- [io-8989-exec-10] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8
2016-02-12 14:38:42.948  INFO 55976 --- [io-8989-exec-10] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8
2016-02-12 14:38:42.948  INFO 55976 --- [ool-37-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:43.002  INFO 55976 --- [io-8989-exec-10] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_daa5d272-b41c-41b4-8430-934ab9d604a8 tuple running: false
2016-02-12 14:38:43.002  INFO 55976 --- [io-8989-exec-10] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StreamingHandler_315f98ad-26db-4ee7-8330-486caab147e9, StaticListener_For_Aggregator]
2016-02-12 14:38:43.002  INFO 55976 --- [io-8989-exec-10] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@4c1882d6]
2016-02-12 14:38:44.196  INFO 55976 --- [nio-8989-exec-1] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:44.196  INFO 55976 --- [nio-8989-exec-1] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:44.196  INFO 55976 --- [nio-8989-exec-1] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:44.196  INFO 55976 --- [nio-8989-exec-1] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:44.196  INFO 55976 --- [nio-8989-exec-1] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_2e71bdda-241f-4105-bd94-d566c2faca9c
2016-02-12 14:38:44.196  INFO 55976 --- [nio-8989-exec-1] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_2e71bdda-241f-4105-bd94-d566c2faca9c
2016-02-12 14:38:44.196  INFO 55976 --- [nio-8989-exec-1] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_2e71bdda-241f-4105-bd94-d566c2faca9c
2016-02-12 14:38:44.197  INFO 55976 --- [nio-8989-exec-1] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_315f98ad-26db-4ee7-8330-486caab147e9, StreamingHandler_2e71bdda-241f-4105-bd94-d566c2faca9c, StaticListener_For_Aggregator]
2016-02-12 14:38:44.197  INFO 55976 --- [nio-8989-exec-1] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:38:46.937  INFO 55976 --- [nio-8989-exec-6] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_315f98ad-26db-4ee7-8330-486caab147e9
2016-02-12 14:38:46.937  INFO 55976 --- [nio-8989-exec-6] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_315f98ad-26db-4ee7-8330-486caab147e9
2016-02-12 14:38:46.937  INFO 55976 --- [ool-39-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:46.990  INFO 55976 --- [nio-8989-exec-6] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_315f98ad-26db-4ee7-8330-486caab147e9 tuple running: false
2016-02-12 14:38:46.991  INFO 55976 --- [nio-8989-exec-6] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StreamingHandler_2e71bdda-241f-4105-bd94-d566c2faca9c, StaticListener_For_Aggregator]
2016-02-12 14:38:46.991  INFO 55976 --- [nio-8989-exec-6] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@50ec080a]
2016-02-12 14:38:50.212  INFO 55976 --- [nio-8989-exec-1] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_2e71bdda-241f-4105-bd94-d566c2faca9c
2016-02-12 14:38:50.212  INFO 55976 --- [nio-8989-exec-1] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_2e71bdda-241f-4105-bd94-d566c2faca9c
2016-02-12 14:38:50.212  INFO 55976 --- [ool-40-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:38:50.265  INFO 55976 --- [nio-8989-exec-1] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_2e71bdda-241f-4105-bd94-d566c2faca9c tuple running: false
2016-02-12 14:38:50.265  INFO 55976 --- [nio-8989-exec-1] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator]
2016-02-12 14:38:50.265  INFO 55976 --- [nio-8989-exec-1] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@160ca0aa]
2016-02-12 14:38:55.051  INFO 55976 --- [nio-8989-exec-7] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:38:55.051  INFO 55976 --- [nio-8989-exec-7] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:38:55.051  INFO 55976 --- [nio-8989-exec-7] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:38:55.051  INFO 55976 --- [nio-8989-exec-7] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:38:55.051  INFO 55976 --- [nio-8989-exec-7] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_278cdeb4-887d-4549-b44d-934afb7241e0
2016-02-12 14:38:55.051  INFO 55976 --- [nio-8989-exec-7] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_278cdeb4-887d-4549-b44d-934afb7241e0
2016-02-12 14:38:55.051  INFO 55976 --- [nio-8989-exec-7] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_278cdeb4-887d-4549-b44d-934afb7241e0
2016-02-12 14:38:55.051  INFO 55976 --- [nio-8989-exec-7] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_278cdeb4-887d-4549-b44d-934afb7241e0, StaticListener_For_Aggregator]
2016-02-12 14:38:55.051  INFO 55976 --- [nio-8989-exec-7] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:39:03.595  INFO 55976 --- [nio-8989-exec-8] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:39:03.595  INFO 55976 --- [nio-8989-exec-8] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:39:03.595  INFO 55976 --- [nio-8989-exec-8] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:39:03.595  INFO 55976 --- [nio-8989-exec-8] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:39:03.595  INFO 55976 --- [nio-8989-exec-8] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_ddc7a2b1-c360-4c78-a045-b45bbd4d9bda
2016-02-12 14:39:03.595  INFO 55976 --- [nio-8989-exec-8] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_ddc7a2b1-c360-4c78-a045-b45bbd4d9bda
2016-02-12 14:39:03.595  INFO 55976 --- [nio-8989-exec-8] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_ddc7a2b1-c360-4c78-a045-b45bbd4d9bda
2016-02-12 14:39:03.596  INFO 55976 --- [nio-8989-exec-8] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_278cdeb4-887d-4549-b44d-934afb7241e0, StaticListener_For_Aggregator, StreamingHandler_ddc7a2b1-c360-4c78-a045-b45bbd4d9bda]
2016-02-12 14:39:03.596  INFO 55976 --- [nio-8989-exec-8] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:39:06.642  INFO 55976 --- [nio-8989-exec-9] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:39:06.643  INFO 55976 --- [nio-8989-exec-9] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:39:06.643  INFO 55976 --- [nio-8989-exec-9] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:39:06.643  INFO 55976 --- [nio-8989-exec-9] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:39:06.643  INFO 55976 --- [nio-8989-exec-9] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_c0d584e5-85f4-4c3e-9569-30c1ae6bd4be
2016-02-12 14:39:06.643  INFO 55976 --- [nio-8989-exec-9] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_c0d584e5-85f4-4c3e-9569-30c1ae6bd4be
2016-02-12 14:39:06.643  INFO 55976 --- [nio-8989-exec-9] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_c0d584e5-85f4-4c3e-9569-30c1ae6bd4be
2016-02-12 14:39:06.643  INFO 55976 --- [nio-8989-exec-9] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_278cdeb4-887d-4549-b44d-934afb7241e0, StaticListener_For_Aggregator, StreamingHandler_c0d584e5-85f4-4c3e-9569-30c1ae6bd4be, StreamingHandler_ddc7a2b1-c360-4c78-a045-b45bbd4d9bda]
2016-02-12 14:39:06.643  INFO 55976 --- [nio-8989-exec-9] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:39:07.065  INFO 55976 --- [nio-8989-exec-7] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_278cdeb4-887d-4549-b44d-934afb7241e0
2016-02-12 14:39:07.065  INFO 55976 --- [nio-8989-exec-7] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_278cdeb4-887d-4549-b44d-934afb7241e0
2016-02-12 14:39:07.066  INFO 55976 --- [ool-41-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:39:07.117  INFO 55976 --- [nio-8989-exec-7] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_278cdeb4-887d-4549-b44d-934afb7241e0 tuple running: false
2016-02-12 14:39:07.117  INFO 55976 --- [nio-8989-exec-7] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator, StreamingHandler_c0d584e5-85f4-4c3e-9569-30c1ae6bd4be, StreamingHandler_ddc7a2b1-c360-4c78-a045-b45bbd4d9bda]
2016-02-12 14:39:07.117  INFO 55976 --- [nio-8989-exec-7] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@7f2e632b]
2016-02-12 14:39:09.601  INFO 55976 --- [nio-8989-exec-8] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_ddc7a2b1-c360-4c78-a045-b45bbd4d9bda
2016-02-12 14:39:09.601  INFO 55976 --- [nio-8989-exec-8] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_ddc7a2b1-c360-4c78-a045-b45bbd4d9bda
2016-02-12 14:39:09.602  INFO 55976 --- [ool-42-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:39:09.656  INFO 55976 --- [nio-8989-exec-8] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_ddc7a2b1-c360-4c78-a045-b45bbd4d9bda tuple running: false
2016-02-12 14:39:09.656  INFO 55976 --- [nio-8989-exec-8] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator, StreamingHandler_c0d584e5-85f4-4c3e-9569-30c1ae6bd4be]
2016-02-12 14:39:09.656  INFO 55976 --- [nio-8989-exec-8] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@50c5c946]
2016-02-12 14:39:12.649  INFO 55976 --- [nio-8989-exec-9] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_c0d584e5-85f4-4c3e-9569-30c1ae6bd4be
2016-02-12 14:39:12.650  INFO 55976 --- [nio-8989-exec-9] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_c0d584e5-85f4-4c3e-9569-30c1ae6bd4be
2016-02-12 14:39:12.650  INFO 55976 --- [ool-43-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:39:12.704  INFO 55976 --- [nio-8989-exec-9] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_c0d584e5-85f4-4c3e-9569-30c1ae6bd4be tuple running: false
2016-02-12 14:39:12.704  INFO 55976 --- [nio-8989-exec-9] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator]
2016-02-12 14:39:12.704  INFO 55976 --- [nio-8989-exec-9] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@6f125481]
2016-02-12 14:39:12.978  INFO 55976 --- [nio-8989-exec-4] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:39:12.978  INFO 55976 --- [nio-8989-exec-4] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:39:12.978  INFO 55976 --- [nio-8989-exec-4] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:39:12.978  INFO 55976 --- [nio-8989-exec-4] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:39:12.979  INFO 55976 --- [nio-8989-exec-4] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_1ea603b9-5cf5-4324-9c33-d91a2c6495d7
2016-02-12 14:39:12.979  INFO 55976 --- [nio-8989-exec-4] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_1ea603b9-5cf5-4324-9c33-d91a2c6495d7
2016-02-12 14:39:12.979  INFO 55976 --- [nio-8989-exec-4] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_1ea603b9-5cf5-4324-9c33-d91a2c6495d7
2016-02-12 14:39:12.979  INFO 55976 --- [nio-8989-exec-4] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_1ea603b9-5cf5-4324-9c33-d91a2c6495d7, StaticListener_For_Aggregator]
2016-02-12 14:39:12.979  INFO 55976 --- [nio-8989-exec-4] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:39:19.676  INFO 55976 --- [nio-8989-exec-5] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:39:19.676  INFO 55976 --- [nio-8989-exec-5] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:39:19.676  INFO 55976 --- [nio-8989-exec-5] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:39:19.676  INFO 55976 --- [nio-8989-exec-5] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:39:19.676  INFO 55976 --- [nio-8989-exec-5] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_be47589e-6855-468f-af2e-8e2ffc161925
2016-02-12 14:39:19.676  INFO 55976 --- [nio-8989-exec-5] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_be47589e-6855-468f-af2e-8e2ffc161925
2016-02-12 14:39:19.676  INFO 55976 --- [nio-8989-exec-5] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_be47589e-6855-468f-af2e-8e2ffc161925
2016-02-12 14:39:19.676  INFO 55976 --- [nio-8989-exec-5] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_1ea603b9-5cf5-4324-9c33-d91a2c6495d7, StaticListener_For_Aggregator, StreamingHandler_be47589e-6855-468f-af2e-8e2ffc161925]
2016-02-12 14:39:19.676  INFO 55976 --- [nio-8989-exec-5] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:39:21.983  INFO 55976 --- [nio-8989-exec-4] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_1ea603b9-5cf5-4324-9c33-d91a2c6495d7
2016-02-12 14:39:21.983  INFO 55976 --- [nio-8989-exec-4] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_1ea603b9-5cf5-4324-9c33-d91a2c6495d7
2016-02-12 14:39:21.983  INFO 55976 --- [ool-44-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:39:22.035  INFO 55976 --- [nio-8989-exec-4] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_1ea603b9-5cf5-4324-9c33-d91a2c6495d7 tuple running: false
2016-02-12 14:39:22.035  INFO 55976 --- [nio-8989-exec-4] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator, StreamingHandler_be47589e-6855-468f-af2e-8e2ffc161925]
2016-02-12 14:39:22.035  INFO 55976 --- [nio-8989-exec-4] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@5fab260a]
2016-02-12 14:39:26.356  INFO 55976 --- [nio-8989-exec-2] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:39:26.356  INFO 55976 --- [nio-8989-exec-2] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:39:26.356  INFO 55976 --- [nio-8989-exec-2] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:39:26.356  INFO 55976 --- [nio-8989-exec-2] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:39:26.356  INFO 55976 --- [nio-8989-exec-2] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_356aae34-4abe-4975-ac96-14ef6bebd204
2016-02-12 14:39:26.356  INFO 55976 --- [nio-8989-exec-2] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_356aae34-4abe-4975-ac96-14ef6bebd204
2016-02-12 14:39:26.356  INFO 55976 --- [nio-8989-exec-2] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_356aae34-4abe-4975-ac96-14ef6bebd204
2016-02-12 14:39:26.356  INFO 55976 --- [nio-8989-exec-2] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StaticListener_For_Aggregator, StreamingHandler_be47589e-6855-468f-af2e-8e2ffc161925, StreamingHandler_356aae34-4abe-4975-ac96-14ef6bebd204]
2016-02-12 14:39:26.356  INFO 55976 --- [nio-8989-exec-2] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:39:28.686  INFO 55976 --- [nio-8989-exec-5] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_be47589e-6855-468f-af2e-8e2ffc161925
2016-02-12 14:39:28.686  INFO 55976 --- [nio-8989-exec-5] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_be47589e-6855-468f-af2e-8e2ffc161925
2016-02-12 14:39:28.686  INFO 55976 --- [ool-45-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:39:28.738  INFO 55976 --- [nio-8989-exec-5] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_be47589e-6855-468f-af2e-8e2ffc161925 tuple running: false
2016-02-12 14:39:28.739  INFO 55976 --- [nio-8989-exec-5] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator, StreamingHandler_356aae34-4abe-4975-ac96-14ef6bebd204]
2016-02-12 14:39:28.739  INFO 55976 --- [nio-8989-exec-5] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@49288c7f]
2016-02-12 14:39:34.059  INFO 55976 --- [nio-8989-exec-3] c.n.t.s.servlet.TurbineStreamServlet     : FilterCriteria: []
2016-02-12 14:39:34.059  INFO 55976 --- [nio-8989-exec-3] c.n.t.s.servlet.TurbineStreamServlet     : StatsType filters: []
2016-02-12 14:39:34.059  INFO 55976 --- [nio-8989-exec-3] c.n.t.s.TurbineStreamingConnection       : Relevance config: []
2016-02-12 14:39:34.059  INFO 55976 --- [nio-8989-exec-3] c.n.t.s.TurbineStreamingConnection       : Relevance metrics config: {}
2016-02-12 14:39:34.060  INFO 55976 --- [nio-8989-exec-3] c.n.t.monitor.cluster.ClusterMonitor     : Registering event handler for cluster monitor: StreamingHandler_40f344d0-f516-41ae-ba8d-65d8691993b7
2016-02-12 14:39:34.060  INFO 55976 --- [nio-8989-exec-3] c.n.t.handler.TurbineDataDispatcher      : 

Just added and starting handler tuple: StreamingHandler_40f344d0-f516-41ae-ba8d-65d8691993b7
2016-02-12 14:39:34.060  INFO 55976 --- [nio-8989-exec-3] c.n.turbine.data.AggDataFromCluster      : Per handler dispacher started for: StreamingHandler_40f344d0-f516-41ae-ba8d-65d8691993b7
2016-02-12 14:39:34.060  INFO 55976 --- [nio-8989-exec-3] c.n.t.monitor.cluster.ClusterMonitor     : All event handlers for cluster monitor: [StreamingHandler_40f344d0-f516-41ae-ba8d-65d8691993b7, StaticListener_For_Aggregator, StreamingHandler_356aae34-4abe-4975-ac96-14ef6bebd204]
2016-02-12 14:39:34.060  INFO 55976 --- [nio-8989-exec-3] c.n.t.monitor.cluster.ClusterMonitor     : Starting up the cluster monitor for AUTH-SERVICE_agg
2016-02-12 14:39:35.366 ERROR 55976 --- [nio-8989-exec-2] c.n.t.s.TurbineStreamingConnection       : Got exception when writing to response stream

org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:393)
    at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:426)
    at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:342)
    at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:317)
    at org.apache.catalina.connector.Response.flushBuffer(Response.java:511)
    at org.apache.catalina.connector.ResponseFacade.flushBuffer(ResponseFacade.java:318)
    at com.netflix.turbine.streaming.servlet.SynchronizedHttpServletResponse.flushBuffer(SynchronizedHttpServletResponse.java:79)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet$ServletStreamHandler.noData(TurbineStreamServlet.java:226)
    at com.netflix.turbine.streaming.TurbineStreamingConnection.waitOnConnection(TurbineStreamingConnection.java:214)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.streamFromCluster(TurbineStreamServlet.java:136)
    at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.doGet(TurbineStreamServlet.java:98)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:300)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:103)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:65)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
    at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:124)
    at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
    at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:172)
    at org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:139)
    at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:197)
    at org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:41)
    at org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:320)
    at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:116)
    at org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:256)
    at org.apache.coyote.Response.doWrite(Response.java:501)
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:388)
    ... 52 common frames omitted

2016-02-12 14:39:36.348  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
2016-02-12 14:39:36.348  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: AUTH-SERVICE
2016-02-12 14:39:36.348  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: AUTH-SERVICE, size=2
2016-02-12 14:39:36.348  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: DEVICE-SERVICE
2016-02-12 14:39:36.348  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: DEVICE-SERVICE, size=2
2016-02-12 14:39:36.348  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Retrieved hosts from InstanceDiscovery: 4
2016-02-12 14:39:36.348  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Found hosts that have been previously terminated: 0
2016-02-12 14:39:36.348  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Hosts up:2, hosts down: 0
2016-02-12 14:39:38.368  INFO 55976 --- [nio-8989-exec-2] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_356aae34-4abe-4975-ac96-14ef6bebd204
2016-02-12 14:39:38.368  INFO 55976 --- [nio-8989-exec-2] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_356aae34-4abe-4975-ac96-14ef6bebd204
2016-02-12 14:39:38.368  INFO 55976 --- [ool-46-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:39:38.421  INFO 55976 --- [nio-8989-exec-2] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_356aae34-4abe-4975-ac96-14ef6bebd204 tuple running: false
2016-02-12 14:39:38.421  INFO 55976 --- [nio-8989-exec-2] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StreamingHandler_40f344d0-f516-41ae-ba8d-65d8691993b7, StaticListener_For_Aggregator]
2016-02-12 14:39:38.422  INFO 55976 --- [nio-8989-exec-2] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@13ec05d3]
2016-02-12 14:39:40.064  INFO 55976 --- [nio-8989-exec-3] c.n.t.s.servlet.TurbineStreamServlet     : 

RETURNING FROM waitOnConnection: StreamingHandler_40f344d0-f516-41ae-ba8d-65d8691993b7
2016-02-12 14:39:40.064  INFO 55976 --- [nio-8989-exec-3] c.n.t.handler.TurbineDataDispatcher      : Removing event handler: StreamingHandler_40f344d0-f516-41ae-ba8d-65d8691993b7
2016-02-12 14:39:40.064  INFO 55976 --- [ool-47-thread-1] com.netflix.turbine.utils.WorkerThread   : Worker got interrupted, shutting down
2016-02-12 14:39:40.119  INFO 55976 --- [nio-8989-exec-3] c.n.turbine.data.AggDataFromCluster      : 

Removing tuple for : StreamingHandler_40f344d0-f516-41ae-ba8d-65d8691993b7 tuple running: false
2016-02-12 14:39:40.119  INFO 55976 --- [nio-8989-exec-3] c.n.t.handler.TurbineDataDispatcher      :  tuples : [StaticListener_For_Aggregator]
2016-02-12 14:39:40.119  INFO 55976 --- [nio-8989-exec-3] c.n.t.handler.TurbineDataDispatcher      : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@22e9166d]
2016-02-12 14:40:36.350  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
2016-02-12 14:40:36.350  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: AUTH-SERVICE
2016-02-12 14:40:36.350  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: AUTH-SERVICE, size=2
2016-02-12 14:40:36.350  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: DEVICE-SERVICE
2016-02-12 14:40:36.350  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: DEVICE-SERVICE, size=2
2016-02-12 14:40:36.350  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Retrieved hosts from InstanceDiscovery: 4
2016-02-12 14:40:36.350  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Found hosts that have been previously terminated: 0
2016-02-12 14:40:36.350  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Hosts up:2, hosts down: 0
2016-02-12 14:41:23.940  INFO 55976 --- [InstanceMonitor] c.n.t.monitor.instance.InstanceMonitor   : no more data from connection to device-service.mi-ae.cn
2016-02-12 14:41:23.940  INFO 55976 --- [InstanceMonitor] c.n.t.monitor.instance.InstanceMonitor   : Re-initing host connection: device-service.mi-ae.cn DEVICE-SERVICE
2016-02-12 14:41:36.354  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
2016-02-12 14:41:36.354  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: AUTH-SERVICE
2016-02-12 14:41:36.354  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: AUTH-SERVICE, size=2
2016-02-12 14:41:36.354  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: DEVICE-SERVICE
2016-02-12 14:41:36.354  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: DEVICE-SERVICE, size=2
2016-02-12 14:41:36.354  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Retrieved hosts from InstanceDiscovery: 4
2016-02-12 14:41:36.354  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Found hosts that have been previously terminated: 0
2016-02-12 14:41:36.354  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Hosts up:2, hosts down: 0
2016-02-12 14:42:36.354  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
2016-02-12 14:42:36.354  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: AUTH-SERVICE
2016-02-12 14:42:36.354  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: AUTH-SERVICE, size=2
2016-02-12 14:42:36.354  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: DEVICE-SERVICE
2016-02-12 14:42:36.354  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: DEVICE-SERVICE, size=2
2016-02-12 14:42:36.354  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Retrieved hosts from InstanceDiscovery: 4
2016-02-12 14:42:36.354  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Found hosts that have been previously terminated: 0
2016-02-12 14:42:36.354  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Hosts up:2, hosts down: 0
2016-02-12 14:43:36.355  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
2016-02-12 14:43:36.355  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: AUTH-SERVICE
2016-02-12 14:43:36.355  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: AUTH-SERVICE, size=2
2016-02-12 14:43:36.355  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: DEVICE-SERVICE
2016-02-12 14:43:36.355  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: DEVICE-SERVICE, size=2
2016-02-12 14:43:36.355  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Retrieved hosts from InstanceDiscovery: 4
2016-02-12 14:43:36.356  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Found hosts that have been previously terminated: 0
2016-02-12 14:43:36.356  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Hosts up:2, hosts down: 0
2016-02-12 14:44:36.355  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
2016-02-12 14:44:36.356  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: AUTH-SERVICE
2016-02-12 14:44:36.356  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: AUTH-SERVICE, size=2
2016-02-12 14:44:36.356  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: DEVICE-SERVICE
2016-02-12 14:44:36.356  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: DEVICE-SERVICE, size=2
2016-02-12 14:44:36.356  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Retrieved hosts from InstanceDiscovery: 4
2016-02-12 14:44:36.356  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Found hosts that have been previously terminated: 0
2016-02-12 14:44:36.356  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Hosts up:2, hosts down: 0
2016-02-12 14:45:36.355  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
2016-02-12 14:45:36.355  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: AUTH-SERVICE
2016-02-12 14:45:36.355  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: AUTH-SERVICE, size=2
2016-02-12 14:45:36.355  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Fetching instances for app: DEVICE-SERVICE
2016-02-12 14:45:36.356  INFO 55976 --- [        Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery  : Received instance list for app: DEVICE-SERVICE, size=2
2016-02-12 14:45:36.356  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Retrieved hosts from InstanceDiscovery: 4
2016-02-12 14:45:36.356  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Found hosts that have been previously terminated: 0
2016-02-12 14:45:36.356  INFO 55976 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Hosts up:2, hosts down: 0

log from cloud turbine server

Fri Feb 12 22:16:27 UTC 2016 [App] STDOUT 2016-02-12 22:16:27.720 INFO 33 --- [o-62188-exec-13] c.n.t.monitor.cluster.ClusterMonitor : All event handlers for cluster monitor: [StreamingHandler_d3424d0e-dd98-46a1-888c-459d3227f7e0, StaticListener_For_Aggregator, StreamingHandler_b1703729-fae9-47e0-bc83-6c9271880f13, StreamingHandler_03c0ba40-482a-4a5d-ae4d-562a212ee0a7, StreamingHandler_3b663afb-3b49-4d61-b73d-03b9cc6220c3, StreamingHandler_f0bae206-3c2b-4cb4-8632-9aaba02ca59e]
Fri Feb 12 22:16:27 UTC 2016 [App] STDOUT 2016-02-12 22:16:27.720 INFO 33 --- [o-62188-exec-13] c.n.t.monitor.cluster.ClusterMonitor : Starting up the cluster monitor for AUTH-SERVICE_agg
Fri Feb 12 22:16:28 UTC 2016 [App] STDOUT 2016-02-12 22:16:28.260 INFO 33 --- [o-62188-exec-11] c.n.t.s.servlet.TurbineStreamServlet : FilterCriteria: []
Fri Feb 12 22:16:28 UTC 2016 [App] STDOUT 2016-02-12 22:16:28.260 INFO 33 --- [o-62188-exec-11] c.n.t.s.servlet.TurbineStreamServlet : StatsType filters: []
Fri Feb 12 22:16:28 UTC 2016 [App] STDOUT 2016-02-12 22:16:28.261 INFO 33 --- [o-62188-exec-11] c.n.t.s.TurbineStreamingConnection : Relevance config: []
Fri Feb 12 22:16:28 UTC 2016 [App] STDOUT 2016-02-12 22:16:28.261 INFO 33 --- [o-62188-exec-11] c.n.t.s.TurbineStreamingConnection : Relevance metrics config: {}
Fri Feb 12 22:16:28 UTC 2016 [App] STDOUT 2016-02-12 22:16:28.261 INFO 33 --- [o-62188-exec-11] c.n.t.monitor.cluster.ClusterMonitor : Registering event handler for cluster monitor: StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a
Fri Feb 12 22:16:28 UTC 2016 [App] STDOUT Just added and starting handler tuple: StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a
Fri Feb 12 22:16:28 UTC 2016 [App] STDOUT 2016-02-12 22:16:28.261 INFO 33 --- [o-62188-exec-11] c.n.t.handler.TurbineDataDispatcher :
Fri Feb 12 22:16:28 UTC 2016 [App] STDOUT 2016-02-12 22:16:28.261 INFO 33 --- [o-62188-exec-11] c.n.turbine.data.AggDataFromCluster : Per handler dispacher started for: StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a
Fri Feb 12 22:16:28 UTC 2016 [App] STDOUT 2016-02-12 22:16:28.261 INFO 33 --- [o-62188-exec-11] c.n.t.monitor.cluster.ClusterMonitor : All event handlers for cluster monitor: [StreamingHandler_d3424d0e-dd98-46a1-888c-459d3227f7e0, StaticListener_For_Aggregator, StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a, StreamingHandler_b1703729-fae9-47e0-bc83-6c9271880f13, StreamingHandler_03c0ba40-482a-4a5d-ae4d-562a212ee0a7, StreamingHandler_3b663afb-3b49-4d61-b73d-03b9cc6220c3, StreamingHandler_f0bae206-3c2b-4cb4-8632-9aaba02ca59e]
Fri Feb 12 22:16:28 UTC 2016 [App] STDOUT 2016-02-12 22:16:28.261 INFO 33 --- [o-62188-exec-11] c.n.t.monitor.cluster.ClusterMonitor : Starting up the cluster monitor for AUTH-SERVICE_agg
Fri Feb 12 22:16:31 UTC 2016 [App] STDOUT 2016-02-12 22:16:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:16:31 UTC 2016 [App] STDOUT 2016-02-12 22:16:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:16:31 UTC 2016 [App] STDOUT 2016-02-12 22:16:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:16:31 UTC 2016 [App] STDOUT 2016-02-12 22:16:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:16:31 UTC 2016 [App] STDOUT 2016-02-12 22:16:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:16:31 UTC 2016 [App] STDOUT 2016-02-12 22:16:31.549 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:16:31 UTC 2016 [App] STDOUT 2016-02-12 22:16:31.549 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:16:31 UTC 2016 [App] STDOUT 2016-02-12 22:16:31.549 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:17:33 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:02:33 +0000] "GET /turbine.stream?cluster=AUTH-SERVICE HTTP/1.1" 200 0 26040 "-" "Apache-HttpClient/4.3.6 (java 1.5)" 10.0.4.213:12586 x_forwarded_for:"54.223.176.248" vcap_request_id:5ef62804-ca3e-4cc3-49d9-a19cc9a0bee7 response_time:900.002276158 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:14:53 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:21:59:53 +0000] "GET /turbine.stream?cluster=AUTH-SERVICE HTTP/1.1" 200 0 26040 "-" "Apache-HttpClient/4.3.6 (java 1.5)" 10.0.4.213:62257 x_forwarded_for:"54.223.176.248" vcap_request_id:c5ef92f7-4d72-43e6-6038-a292ebf7b475 response_time:900.001265031 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:393)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:426)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:342)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:317)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.connector.Response.flushBuffer(Response.java:511)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.connector.ResponseFacade.flushBuffer(ResponseFacade.java:318)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at com.netflix.turbine.streaming.servlet.SynchronizedHttpServletResponse.flushBuffer(SynchronizedHttpServletResponse.java:79)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at com.netflix.turbine.streaming.servlet.TurbineStreamServlet$ServletStreamHandler.noData(TurbineStreamServlet.java:226)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at com.netflix.turbine.streaming.TurbineStreamingConnection.waitOnConnection(TurbineStreamingConnection.java:214)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.streamFromCluster(TurbineStreamServlet.java:136)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.doGet(TurbineStreamServlet.java:98)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:300)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:103)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:77)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT Caused by: java.io.IOException: Broken pipe
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at java.lang.Thread.run(Thread.java:745)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at sun.nio.ch.IOUtil.write(IOUtil.java:65)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:197)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:172)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:139)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:41)
Fri Feb 12 22:16:54 UTC 2016 [App] STDOUT at org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:320)
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT 2016-02-12 22:16:57.768 INFO 33 --- [o-62188-exec-19] c.n.t.s.servlet.TurbineStreamServlet :
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT 2016-02-12 22:16:57.768 INFO 33 --- [o-62188-exec-15] c.n.t.s.servlet.TurbineStreamServlet :
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT 2016-02-12 22:16:57.769 INFO 33 --- [o-62188-exec-15] c.n.t.handler.TurbineDataDispatcher : Removing event handler: StreamingHandler_b1703729-fae9-47e0-bc83-6c9271880f13
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT 2016-02-12 22:16:57.768 INFO 33 --- [o-62188-exec-19] c.n.t.handler.TurbineDataDispatcher : Removing event handler: StreamingHandler_3b663afb-3b49-4d61-b73d-03b9cc6220c3
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT RETURNING FROM waitOnConnection: StreamingHandler_b1703729-fae9-47e0-bc83-6c9271880f13
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT 2016-02-12 22:16:57.769 INFO 33 --- [ol-143-thread-1] com.netflix.turbine.utils.WorkerThread : Worker got interrupted, shutting down
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT Removing tuple for : StreamingHandler_3b663afb-3b49-4d61-b73d-03b9cc6220c3 tuple running: false
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT 2016-02-12 22:16:57.819 INFO 33 --- [o-62188-exec-19] c.n.turbine.data.AggDataFromCluster :
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT Removing tuple for : StreamingHandler_b1703729-fae9-47e0-bc83-6c9271880f13 tuple running: false
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT 2016-02-12 22:16:57.819 INFO 33 --- [o-62188-exec-19] c.n.t.handler.TurbineDataDispatcher : tuples : [StreamingHandler_d3424d0e-dd98-46a1-888c-459d3227f7e0, StaticListener_For_Aggregator, StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a, StreamingHandler_b1703729-fae9-47e0-bc83-6c9271880f13, StreamingHandler_03c0ba40-482a-4a5d-ae4d-562a212ee0a7, StreamingHandler_f0bae206-3c2b-4cb4-8632-9aaba02ca59e]
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT 2016-02-12 22:16:57.819 INFO 33 --- [o-62188-exec-15] c.n.t.handler.TurbineDataDispatcher : tuples : [StreamingHandler_d3424d0e-dd98-46a1-888c-459d3227f7e0, StaticListener_For_Aggregator, StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a, StreamingHandler_03c0ba40-482a-4a5d-ae4d-562a212ee0a7, StreamingHandler_f0bae206-3c2b-4cb4-8632-9aaba02ca59e]
Fri Feb 12 22:16:57 UTC 2016 [App] STDOUT 2016-02-12 22:16:57.819 INFO 33 --- [o-62188-exec-19] c.n.t.handler.TurbineDataDispatcher : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@1f876f03]
Fri Feb 12 22:17:31 UTC 2016 [App] STDOUT 2016-02-12 22:17:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:17:31 UTC 2016 [App] STDOUT 2016-02-12 22:17:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:17:31 UTC 2016 [App] STDOUT 2016-02-12 22:17:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:17:31 UTC 2016 [App] STDOUT 2016-02-12 22:17:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:17:31 UTC 2016 [App] STDOUT 2016-02-12 22:17:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:17:31 UTC 2016 [App] STDOUT 2016-02-12 22:17:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:18:31 UTC 2016 [App] STDOUT 2016-02-12 22:18:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:18:31 UTC 2016 [App] STDOUT 2016-02-12 22:18:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:18:31 UTC 2016 [App] STDOUT 2016-02-12 22:18:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:18:31 UTC 2016 [App] STDOUT 2016-02-12 22:18:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:18:31 UTC 2016 [App] STDOUT 2016-02-12 22:18:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:18:31 UTC 2016 [App] STDOUT 2016-02-12 22:18:31.549 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:19:31 UTC 2016 [App] STDOUT 2016-02-12 22:19:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:19:31 UTC 2016 [App] STDOUT 2016-02-12 22:19:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:19:31 UTC 2016 [App] STDOUT 2016-02-12 22:19:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:19:31 UTC 2016 [App] STDOUT 2016-02-12 22:19:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:19:31 UTC 2016 [App] STDOUT 2016-02-12 22:19:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:20:31 UTC 2016 [App] STDOUT 2016-02-12 22:20:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:20:31 UTC 2016 [App] STDOUT 2016-02-12 22:20:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:20:31 UTC 2016 [App] STDOUT 2016-02-12 22:20:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:20:31 UTC 2016 [App] STDOUT 2016-02-12 22:20:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:20:31 UTC 2016 [App] STDOUT 2016-02-12 22:20:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:20:31 UTC 2016 [App] STDOUT 2016-02-12 22:20:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:21:24 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:06:24 +0000] "GET /turbine.stream?cluster=AUTH-SERVICE HTTP/1.1" 200 0 25945 "-" "Apache-HttpClient/4.3.6 (java 1.5)" 10.0.4.213:12786 x_forwarded_for:"54.223.176.248" vcap_request_id:f6fb8d61-5169-45c0-5cb7-97350557f12a response_time:900.001853431 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:18:45 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:03:45 +0000] "GET /turbine.stream?cluster=AUTH-SERVICE HTTP/1.1" 200 0 25945 "-" "Apache-HttpClient/4.3.6 (java 1.5)" 10.0.4.213:62458 x_forwarded_for:"54.223.176.248" vcap_request_id:a081cdcb-c970-4289-4b1a-79e62f5b77da response_time:900.000627880 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:20:46 UTC 2016 [App] STDOUT 2016-02-12 22:20:46.137 INFO 33 --- [o-62188-exec-20] c.n.t.s.servlet.TurbineStreamServlet :
Fri Feb 12 22:20:46 UTC 2016 [App] STDOUT 2016-02-12 22:20:46.137 INFO 33 --- [o-62188-exec-20] c.n.t.handler.TurbineDataDispatcher : Removing event handler: StreamingHandler_d3424d0e-dd98-46a1-888c-459d3227f7e0
Fri Feb 12 22:20:46 UTC 2016 [App] STDOUT 2016-02-12 22:20:46.138 INFO 33 --- [o-62188-exec-21] c.n.t.s.servlet.TurbineStreamServlet :
Fri Feb 12 22:20:46 UTC 2016 [App] STDOUT 2016-02-12 22:20:46.138 INFO 33 --- [o-62188-exec-21] c.n.t.handler.TurbineDataDispatcher : Removing event handler: StreamingHandler_03c0ba40-482a-4a5d-ae4d-562a212ee0a7
Fri Feb 12 22:20:46 UTC 2016 [App] STDOUT 2016-02-12 22:20:46.138 INFO 33 --- [ol-145-thread-1] com.netflix.turbine.utils.WorkerThread : Worker got interrupted, shutting down
Fri Feb 12 22:20:46 UTC 2016 [App] STDOUT Removing tuple for : StreamingHandler_d3424d0e-dd98-46a1-888c-459d3227f7e0 tuple running: false
Fri Feb 12 22:20:46 UTC 2016 [App] STDOUT 2016-02-12 22:20:46.187 INFO 33 --- [o-62188-exec-20] c.n.turbine.data.AggDataFromCluster :
Fri Feb 12 22:20:46 UTC 2016 [App] STDOUT 2016-02-12 22:20:46.187 INFO 33 --- [o-62188-exec-20] c.n.t.handler.TurbineDataDispatcher : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@f7c714b]
Fri Feb 12 22:20:46 UTC 2016 [App] STDOUT 2016-02-12 22:20:46.188 INFO 33 --- [o-62188-exec-21] c.n.turbine.data.AggDataFromCluster :
Fri Feb 12 22:20:46 UTC 2016 [App] STDOUT Removing tuple for : StreamingHandler_03c0ba40-482a-4a5d-ae4d-562a212ee0a7 tuple running: false
Fri Feb 12 22:20:46 UTC 2016 [App] STDOUT 2016-02-12 22:20:46.188 INFO 33 --- [o-62188-exec-21] c.n.t.handler.TurbineDataDispatcher : tuples : [StaticListener_For_Aggregator, StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a, StreamingHandler_f0bae206-3c2b-4cb4-8632-9aaba02ca59e]
Fri Feb 12 22:20:48 UTC 2016 [App] STDOUT 2016-02-12 22:20:48.861 INFO 33 --- [o-62188-exec-24] c.n.t.s.servlet.TurbineStreamServlet : FilterCriteria: []
Fri Feb 12 22:20:48 UTC 2016 [App] STDOUT 2016-02-12 22:20:48.861 INFO 33 --- [o-62188-exec-24] c.n.t.s.servlet.TurbineStreamServlet : StatsType filters: []
Fri Feb 12 22:20:48 UTC 2016 [App] STDOUT 2016-02-12 22:20:48.861 INFO 33 --- [o-62188-exec-24] c.n.t.s.TurbineStreamingConnection : Relevance config: []
Fri Feb 12 22:20:48 UTC 2016 [App] STDOUT 2016-02-12 22:20:48.861 INFO 33 --- [o-62188-exec-24] c.n.t.s.TurbineStreamingConnection : Relevance metrics config: {}
Fri Feb 12 22:20:48 UTC 2016 [App] STDOUT 2016-02-12 22:20:48.861 INFO 33 --- [o-62188-exec-24] c.n.t.monitor.cluster.ClusterMonitor : Registering event handler for cluster monitor: StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917
Fri Feb 12 22:20:48 UTC 2016 [App] STDOUT Just added and starting handler tuple: StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917
Fri Feb 12 22:20:48 UTC 2016 [App] STDOUT 2016-02-12 22:20:48.861 INFO 33 --- [o-62188-exec-24] c.n.t.handler.TurbineDataDispatcher :
Fri Feb 12 22:20:48 UTC 2016 [App] STDOUT 2016-02-12 22:20:48.862 INFO 33 --- [o-62188-exec-24] c.n.turbine.data.AggDataFromCluster : Per handler dispacher started for: StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917
Fri Feb 12 22:20:48 UTC 2016 [App] STDOUT 2016-02-12 22:20:48.862 INFO 33 --- [o-62188-exec-24] c.n.t.monitor.cluster.ClusterMonitor : All event handlers for cluster monitor: [StaticListener_For_Aggregator, StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a, StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917, StreamingHandler_f0bae206-3c2b-4cb4-8632-9aaba02ca59e]
Fri Feb 12 22:20:49 UTC 2016 [App] STDOUT 2016-02-12 22:20:49.528 INFO 33 --- [o-62188-exec-12] c.n.t.s.servlet.TurbineStreamServlet : StatsType filters: []
Fri Feb 12 22:20:49 UTC 2016 [App] STDOUT 2016-02-12 22:20:49.528 INFO 33 --- [o-62188-exec-12] c.n.t.s.servlet.TurbineStreamServlet : FilterCriteria: []
Fri Feb 12 22:20:49 UTC 2016 [App] STDOUT 2016-02-12 22:20:49.528 INFO 33 --- [o-62188-exec-12] c.n.t.s.TurbineStreamingConnection : Relevance config: []
Fri Feb 12 22:20:49 UTC 2016 [App] STDOUT 2016-02-12 22:20:49.528 INFO 33 --- [o-62188-exec-12] c.n.t.s.TurbineStreamingConnection : Relevance metrics config: {}
Fri Feb 12 22:20:49 UTC 2016 [App] STDOUT 2016-02-12 22:20:49.528 INFO 33 --- [o-62188-exec-12] c.n.t.monitor.cluster.ClusterMonitor : Registering event handler for cluster monitor: StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854
Fri Feb 12 22:20:49 UTC 2016 [App] STDOUT 2016-02-12 22:20:49.528 INFO 33 --- [o-62188-exec-12] c.n.t.handler.TurbineDataDispatcher :
Fri Feb 12 22:20:49 UTC 2016 [App] STDOUT Just added and starting handler tuple: StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854
Fri Feb 12 22:20:49 UTC 2016 [App] STDOUT 2016-02-12 22:20:49.528 INFO 33 --- [o-62188-exec-12] c.n.turbine.data.AggDataFromCluster : Per handler dispacher started for: StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854
Fri Feb 12 22:20:49 UTC 2016 [App] STDOUT 2016-02-12 22:20:49.528 INFO 33 --- [o-62188-exec-12] c.n.t.monitor.cluster.ClusterMonitor : All event handlers for cluster monitor: [StaticListener_For_Aggregator, StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a, StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917, StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854, StreamingHandler_f0bae206-3c2b-4cb4-8632-9aaba02ca59e]
Fri Feb 12 22:20:49 UTC 2016 [App] STDOUT 2016-02-12 22:20:49.528 INFO 33 --- [o-62188-exec-12] c.n.t.monitor.cluster.ClusterMonitor : Starting up the cluster monitor for AUTH-SERVICE_agg
Fri Feb 12 22:21:31 UTC 2016 [App] STDOUT 2016-02-12 22:21:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:21:31 UTC 2016 [App] STDOUT 2016-02-12 22:21:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:21:31 UTC 2016 [App] STDOUT 2016-02-12 22:21:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:21:31 UTC 2016 [App] STDOUT 2016-02-12 22:21:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:21:31 UTC 2016 [App] STDOUT 2016-02-12 22:21:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:21:31 UTC 2016 [App] STDOUT 2016-02-12 22:21:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:22:31 UTC 2016 [App] STDOUT 2016-02-12 22:22:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:22:31 UTC 2016 [App] STDOUT 2016-02-12 22:22:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:22:31 UTC 2016 [App] STDOUT 2016-02-12 22:22:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:22:31 UTC 2016 [App] STDOUT 2016-02-12 22:22:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:22:31 UTC 2016 [App] STDOUT 2016-02-12 22:22:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:22:31 UTC 2016 [App] STDOUT 2016-02-12 22:22:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:23:31 UTC 2016 [App] STDOUT 2016-02-12 22:23:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:23:31 UTC 2016 [App] STDOUT 2016-02-12 22:23:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:23:31 UTC 2016 [App] STDOUT 2016-02-12 22:23:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:23:31 UTC 2016 [App] STDOUT 2016-02-12 22:23:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:23:31 UTC 2016 [App] STDOUT 2016-02-12 22:23:31.549 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:23:31 UTC 2016 [App] STDOUT 2016-02-12 22:23:31.549 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:24:31 UTC 2016 [App] STDOUT 2016-02-12 22:24:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:24:31 UTC 2016 [App] STDOUT 2016-02-12 22:24:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:24:31 UTC 2016 [App] STDOUT 2016-02-12 22:24:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:24:31 UTC 2016 [App] STDOUT 2016-02-12 22:24:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:24:31 UTC 2016 [App] STDOUT 2016-02-12 22:24:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:24:31 UTC 2016 [App] STDOUT 2016-02-12 22:24:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:25:31 UTC 2016 [App] STDOUT 2016-02-12 22:25:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:25:31 UTC 2016 [App] STDOUT 2016-02-12 22:25:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:25:31 UTC 2016 [App] STDOUT 2016-02-12 22:25:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:25:31 UTC 2016 [App] STDOUT 2016-02-12 22:25:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:25:31 UTC 2016 [App] STDOUT 2016-02-12 22:25:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:25:32 UTC 2016 [App] STDOUT 2016-02-12 22:25:32.234 INFO 33 --- [InstanceMonitor] c.n.t.monitor.instance.InstanceMonitor : no more data from connection to device-service.mi-ae.cn
Fri Feb 12 22:25:32 UTC 2016 [App] STDOUT 2016-02-12 22:25:32.234 INFO 33 --- [InstanceMonitor] c.n.t.monitor.instance.InstanceMonitor : Re-initing host connection: device-service.mi-ae.cn DEVICE-SERVICE
Fri Feb 12 22:26:31 UTC 2016 [App] STDOUT 2016-02-12 22:26:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:26:31 UTC 2016 [App] STDOUT 2016-02-12 22:26:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:26:31 UTC 2016 [App] STDOUT 2016-02-12 22:26:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:26:31 UTC 2016 [App] STDOUT 2016-02-12 22:26:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:26:31 UTC 2016 [App] STDOUT 2016-02-12 22:26:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:27:31 UTC 2016 [App] STDOUT 2016-02-12 22:27:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:27:31 UTC 2016 [App] STDOUT 2016-02-12 22:27:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:27:31 UTC 2016 [App] STDOUT 2016-02-12 22:27:31.548 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:27:31 UTC 2016 [App] STDOUT 2016-02-12 22:27:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:27:31 UTC 2016 [App] STDOUT 2016-02-12 22:27:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:27:31 UTC 2016 [App] STDOUT 2016-02-12 22:27:31.548 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:27:31 UTC 2016 [App] STDOUT 2016-02-12 22:27:31.633 INFO 33 --- [InstanceMonitor] c.n.t.monitor.instance.InstanceMonitor : no more data from connection to auth-service.mi-ae.cn
Fri Feb 12 22:28:23 UTC 2016 [App] STDOUT 2016-02-12 22:28:23.035 DEBUG 33 --- [o-62188-exec-16] o.s.b.a.e.mvc.EndpointHandlerMapping : Looking up handler method for path /
Fri Feb 12 22:28:23 UTC 2016 [App] STDOUT 2016-02-12 22:28:23.036 DEBUG 33 --- [o-62188-exec-16] o.s.b.a.e.mvc.EndpointHandlerMapping : Looking up handler method for path /error
Fri Feb 12 22:26:25 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:26:25 +0000] "GET / HTTP/1.1" 404 0 104 "-" "Python-urllib/2.7" 10.0.3.194:3254 x_forwarded_for:"106.120.203.232" vcap_request_id:a44b6924-5830-407d-581a-04beb9ef5702 response_time:0.002917685 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:28:26 UTC 2016 [App] STDOUT 2016-02-12 22:28:26.521 DEBUG 33 --- [o-62188-exec-19] o.s.b.a.e.mvc.EndpointHandlerMapping : Looking up handler method for path /
Fri Feb 12 22:28:26 UTC 2016 [App] STDOUT 2016-02-12 22:28:26.521 DEBUG 33 --- [o-62188-exec-19] o.s.b.a.e.mvc.EndpointHandlerMapping : Did not find handler method for [/]
Fri Feb 12 22:28:26 UTC 2016 [App] STDOUT 2016-02-12 22:28:26.522 DEBUG 33 --- [o-62188-exec-19] o.s.b.a.e.mvc.EndpointHandlerMapping : Did not find handler method for [/error]
Fri Feb 12 22:29:08 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:29:08 +0000] "GET / HTTP/1.1" 404 0 104 "-" "Python-urllib/2.7" 10.0.3.194:15518 x_forwarded_for:"106.120.203.232" vcap_request_id:7758566f-e8d5-44c7-7279-455fcf505f22 response_time:0.005273183 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:28:31 UTC 2016 [App] STDOUT 2016-02-12 22:28:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:28:31 UTC 2016 [App] STDOUT 2016-02-12 22:28:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:28:31 UTC 2016 [App] STDOUT 2016-02-12 22:28:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:28:31 UTC 2016 [App] STDOUT 2016-02-12 22:28:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:28:31 UTC 2016 [App] STDOUT 2016-02-12 22:28:31.549 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:28:31 UTC 2016 [App] STDOUT 2016-02-12 22:28:31.549 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:28:31 UTC 2016 [App] STDOUT 2016-02-12 22:28:31.549 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:29:31 UTC 2016 [App] STDOUT 2016-02-12 22:29:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:29:31 UTC 2016 [App] STDOUT 2016-02-12 22:29:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:29:31 UTC 2016 [App] STDOUT 2016-02-12 22:29:31.549 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:29:31 UTC 2016 [App] STDOUT 2016-02-12 22:29:31.549 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:29:31 UTC 2016 [App] STDOUT 2016-02-12 22:29:31.549 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:30:31 UTC 2016 [App] STDOUT 2016-02-12 22:30:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:30:31 UTC 2016 [App] STDOUT 2016-02-12 22:30:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:30:31 UTC 2016 [App] STDOUT 2016-02-12 22:30:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:30:31 UTC 2016 [App] STDOUT 2016-02-12 22:30:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:30:31 UTC 2016 [App] STDOUT 2016-02-12 22:30:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:30:31 UTC 2016 [App] STDOUT 2016-02-12 22:30:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:29:29 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:14:29 +0000] "GET /turbine.stream?cluster=AUTH-SERVICE HTTP/1.1" 200 0 25945 "-" "Apache-HttpClient/4.3.6 (java 1.5)" 10.0.4.213:63057 x_forwarded_for:"54.223.176.248" vcap_request_id:6a3761da-bd9c-443a-4c0c-816f027e9ba1 response_time:900.000700416 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:29:30 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:14:30 +0000] "GET /turbine.stream?cluster=AUTH-SERVICE HTTP/1.1" 200 0 26040 "-" "Apache-HttpClient/4.3.6 (java 1.5)" 10.0.4.213:62219 x_forwarded_for:"54.223.176.248" vcap_request_id:04c44843-25fb-4d3c-6b2f-eeca818d0189 response_time:900.001935695 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:31:30 UTC 2016 [App] STDOUT 2016-02-12 22:31:30.796 INFO 33 --- [o-62188-exec-13] c.n.t.s.servlet.TurbineStreamServlet :
Fri Feb 12 22:31:30 UTC 2016 [App] STDOUT 2016-02-12 22:31:30.796 INFO 33 --- [o-62188-exec-13] c.n.t.handler.TurbineDataDispatcher : Removing event handler: StreamingHandler_f0bae206-3c2b-4cb4-8632-9aaba02ca59e
Fri Feb 12 22:31:30 UTC 2016 [App] STDOUT 2016-02-12 22:31:30.846 INFO 33 --- [o-62188-exec-13] c.n.turbine.data.AggDataFromCluster :
Fri Feb 12 22:31:30 UTC 2016 [App] STDOUT 2016-02-12 22:31:30.846 INFO 33 --- [o-62188-exec-13] c.n.t.handler.TurbineDataDispatcher : tuples : [StaticListener_For_Aggregator, StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a, StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917, StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854]
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT 2016-02-12 22:31:31.343 ERROR 33 --- [o-62188-exec-11] c.n.t.s.TurbineStreamingConnection : Got exception when writing to response stream
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:393)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:426)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:342)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:317)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.connector.Response.flushBuffer(Response.java:511)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.connector.ResponseFacade.flushBuffer(ResponseFacade.java:318)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at com.netflix.turbine.streaming.servlet.TurbineStreamServlet$ServletStreamHandler.noData(TurbineStreamServlet.java:226)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at com.netflix.turbine.streaming.servlet.SynchronizedHttpServletResponse.flushBuffer(SynchronizedHttpServletResponse.java:79)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at com.netflix.turbine.streaming.TurbineStreamingConnection.waitOnConnection(TurbineStreamingConnection.java:214)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.streamFromCluster(TurbineStreamServlet.java:136)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at com.netflix.turbine.streaming.servlet.TurbineStreamServlet.doGet(TurbineStreamServlet.java:98)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:300)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:103)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:77)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at java.lang.Thread.run(Thread.java:745)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT Caused by: java.io.IOException: Broken pipe
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at sun.nio.ch.IOUtil.write(IOUtil.java:65)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:124)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:172)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:139)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:41)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:320)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:116)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:256)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.coyote.Response.doWrite(Response.java:501)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:388)
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT 2016-02-12 22:31:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT 2016-02-12 22:31:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT 2016-02-12 22:31:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT 2016-02-12 22:31:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:31:31 UTC 2016 [App] STDOUT 2016-02-12 22:31:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT 2016-02-12 22:31:33.493 INFO 33 --- [o-62188-exec-20] c.n.t.s.servlet.TurbineStreamServlet : FilterCriteria: []
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT 2016-02-12 22:31:33.494 INFO 33 --- [o-62188-exec-20] c.n.t.s.servlet.TurbineStreamServlet : StatsType filters: []
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT 2016-02-12 22:31:33.494 INFO 33 --- [o-62188-exec-20] c.n.t.s.TurbineStreamingConnection : Relevance config: []
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT 2016-02-12 22:31:33.494 INFO 33 --- [o-62188-exec-20] c.n.t.monitor.cluster.ClusterMonitor : Registering event handler for cluster monitor: StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT 2016-02-12 22:31:33.494 INFO 33 --- [o-62188-exec-20] c.n.t.handler.TurbineDataDispatcher :
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT Just added and starting handler tuple: StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT 2016-02-12 22:31:33.494 INFO 33 --- [o-62188-exec-20] c.n.turbine.data.AggDataFromCluster : Per handler dispacher started for: StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT 2016-02-12 22:31:33.494 INFO 33 --- [o-62188-exec-20] c.n.t.monitor.cluster.ClusterMonitor : All event handlers for cluster monitor: [StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e, StaticListener_For_Aggregator, StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a, StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917, StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854]
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT 2016-02-12 22:31:33.583 INFO 33 --- [o-62188-exec-21] c.n.t.s.servlet.TurbineStreamServlet : FilterCriteria: []
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT 2016-02-12 22:31:33.583 INFO 33 --- [o-62188-exec-21] c.n.t.s.TurbineStreamingConnection : Relevance metrics config: {}
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT 2016-02-12 22:31:33.583 INFO 33 --- [o-62188-exec-21] c.n.t.monitor.cluster.ClusterMonitor : Registering event handler for cluster monitor: StreamingHandler_9cda3aca-52d7-4594-9bcb-38538e89c46e
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT Just added and starting handler tuple: StreamingHandler_9cda3aca-52d7-4594-9bcb-38538e89c46e
Fri Feb 12 22:31:33 UTC 2016 [App] STDOUT 2016-02-12 22:31:33.584 INFO 33 --- [o-62188-exec-21] c.n.t.monitor.cluster.ClusterMonitor : All event handlers for cluster monitor: [StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e, StaticListener_For_Aggregator, StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a, StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917, StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854, StreamingHandler_9cda3aca-52d7-4594-9bcb-38538e89c46e]
Fri Feb 12 22:31:34 UTC 2016 [App] STDOUT 2016-02-12 22:31:34.343 INFO 33 --- [o-62188-exec-11] c.n.t.s.servlet.TurbineStreamServlet :
Fri Feb 12 22:31:34 UTC 2016 [App] STDOUT 2016-02-12 22:31:34.343 INFO 33 --- [o-62188-exec-11] c.n.t.handler.TurbineDataDispatcher : Removing event handler: StreamingHandler_f89e5ca1-b765-4d56-9fc0-454fb51ef62a
Fri Feb 12 22:31:34 UTC 2016 [App] STDOUT 2016-02-12 22:31:34.394 INFO 33 --- [o-62188-exec-11] c.n.turbine.data.AggDataFromCluster :
Fri Feb 12 22:31:34 UTC 2016 [App] STDOUT 2016-02-12 22:31:34.394 INFO 33 --- [o-62188-exec-11] c.n.t.handler.TurbineDataDispatcher : tuples : [StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e, StaticListener_For_Aggregator, StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917, StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854, StreamingHandler_9cda3aca-52d7-4594-9bcb-38538e89c46e]
Fri Feb 12 22:31:34 UTC 2016 [App] STDOUT 2016-02-12 22:31:34.394 INFO 33 --- [o-62188-exec-11] c.n.t.handler.TurbineDataDispatcher : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@20cab989]
Fri Feb 12 22:32:31 UTC 2016 [App] STDOUT 2016-02-12 22:32:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:32:31 UTC 2016 [App] STDOUT 2016-02-12 22:32:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:32:31 UTC 2016 [App] STDOUT 2016-02-12 22:32:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:32:31 UTC 2016 [App] STDOUT 2016-02-12 22:32:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:32:31 UTC 2016 [App] STDOUT 2016-02-12 22:32:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:32:31 UTC 2016 [App] STDOUT 2016-02-12 22:32:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:32:31 UTC 2016 [App] STDOUT 2016-02-12 22:32:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:33:31 UTC 2016 [App] STDOUT 2016-02-12 22:33:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:33:31 UTC 2016 [App] STDOUT 2016-02-12 22:33:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:33:31 UTC 2016 [App] STDOUT 2016-02-12 22:33:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:33:31 UTC 2016 [App] STDOUT 2016-02-12 22:33:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:33:31 UTC 2016 [App] STDOUT 2016-02-12 22:33:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:33:31 UTC 2016 [App] STDOUT 2016-02-12 22:33:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:33:31 UTC 2016 [App] STDOUT 2016-02-12 22:33:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:33:53 UTC 2016 [App] STDOUT 2016-02-12 22:33:53.812 INFO 33 --- [InstanceMonitor] c.n.t.monitor.instance.InstanceMonitor : no more data from connection to auth-service.mi-ae.cn
Fri Feb 12 22:34:31 UTC 2016 [App] STDOUT 2016-02-12 22:34:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:34:31 UTC 2016 [App] STDOUT 2016-02-12 22:34:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:34:31 UTC 2016 [App] STDOUT 2016-02-12 22:34:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:34:31 UTC 2016 [App] STDOUT 2016-02-12 22:34:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:34:31 UTC 2016 [App] STDOUT 2016-02-12 22:34:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:34:31 UTC 2016 [App] STDOUT 2016-02-12 22:34:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:34:31 UTC 2016 [App] STDOUT 2016-02-12 22:34:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:35:31 UTC 2016 [App] STDOUT 2016-02-12 22:35:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:35:31 UTC 2016 [App] STDOUT 2016-02-12 22:35:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=3
Fri Feb 12 22:35:31 UTC 2016 [App] STDOUT 2016-02-12 22:35:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:35:31 UTC 2016 [App] STDOUT 2016-02-12 22:35:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:35:31 UTC 2016 [App] STDOUT 2016-02-12 22:35:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:35:31 UTC 2016 [App] STDOUT 2016-02-12 22:35:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 5
Fri Feb 12 22:35:31 UTC 2016 [App] STDOUT 2016-02-12 22:35:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:35:31 UTC 2016 [App] STDOUT 2016-02-12 22:35:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:33:51 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:18:51 +0000] "GET /turbine.stream?cluster=AUTH-SERVICE HTTP/1.1" 200 0 25945 "-" "Apache-HttpClient/4.3.6 (java 1.5)" 10.0.4.213:63298 x_forwarded_for:"54.223.176.248" vcap_request_id:687b4f52-eb16-4816-6317-b8463ba27953 response_time:900.001105443 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:33:51 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:18:51 +0000] "GET /turbine.stream?cluster=AUTH-SERVICE HTTP/1.1" 200 0 25945 "-" "Apache-HttpClient/4.3.6 (java 1.5)" 10.0.4.213:62458 x_forwarded_for:"54.223.176.248" vcap_request_id:eec3fce3-6c17-4fe6-7adb-5ff8cf43b965 response_time:900.001680608 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:35:51 UTC 2016 [App] STDOUT 2016-02-12 22:35:51.938 INFO 33 --- [o-62188-exec-24] c.n.t.s.servlet.TurbineStreamServlet :
Fri Feb 12 22:35:51 UTC 2016 [App] STDOUT RETURNING FROM waitOnConnection: StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917
Fri Feb 12 22:35:51 UTC 2016 [App] STDOUT 2016-02-12 22:35:51.938 INFO 33 --- [o-62188-exec-24] c.n.t.handler.TurbineDataDispatcher : Removing event handler: StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917
Fri Feb 12 22:35:51 UTC 2016 [App] STDOUT 2016-02-12 22:35:51.938 INFO 33 --- [ol-148-thread-1] com.netflix.turbine.utils.WorkerThread : Worker got interrupted, shutting down
Fri Feb 12 22:35:51 UTC 2016 [App] STDOUT Removing tuple for : StreamingHandler_87a6a88e-75e4-4d32-962f-2cdcafeb2917 tuple running: false
Fri Feb 12 22:35:51 UTC 2016 [App] STDOUT 2016-02-12 22:35:51.989 INFO 33 --- [o-62188-exec-24] c.n.t.handler.TurbineDataDispatcher : tuples : [StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e, StaticListener_For_Aggregator, StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854, StreamingHandler_9cda3aca-52d7-4594-9bcb-38538e89c46e]
Fri Feb 12 22:35:51 UTC 2016 [App] STDOUT 2016-02-12 22:35:51.989 INFO 33 --- [o-62188-exec-24] c.n.t.handler.TurbineDataDispatcher : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@1f1bc03a]
Fri Feb 12 22:35:51 UTC 2016 [App] STDOUT 2016-02-12 22:35:51.989 INFO 33 --- [o-62188-exec-24] c.n.turbine.data.AggDataFromCluster :
Fri Feb 12 22:35:52 UTC 2016 [App] STDOUT 2016-02-12 22:35:52.604 INFO 33 --- [o-62188-exec-12] c.n.t.s.servlet.TurbineStreamServlet :
Fri Feb 12 22:35:52 UTC 2016 [App] STDOUT RETURNING FROM waitOnConnection: StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854
Fri Feb 12 22:35:52 UTC 2016 [App] STDOUT 2016-02-12 22:35:52.604 INFO 33 --- [o-62188-exec-12] c.n.t.handler.TurbineDataDispatcher : Removing event handler: StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854
Fri Feb 12 22:35:52 UTC 2016 [App] STDOUT 2016-02-12 22:35:52.604 INFO 33 --- [ol-149-thread-1] com.netflix.turbine.utils.WorkerThread : Worker got interrupted, shutting down
Fri Feb 12 22:35:52 UTC 2016 [App] STDOUT Removing tuple for : StreamingHandler_33a10fbf-c353-4fbc-9be7-7fd666d0f854 tuple running: false
Fri Feb 12 22:35:52 UTC 2016 [App] STDOUT 2016-02-12 22:35:52.654 INFO 33 --- [o-62188-exec-12] c.n.t.handler.TurbineDataDispatcher : tuples : [StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e, StaticListener_For_Aggregator, StreamingHandler_9cda3aca-52d7-4594-9bcb-38538e89c46e]
Fri Feb 12 22:35:52 UTC 2016 [App] STDOUT 2016-02-12 22:35:52.654 INFO 33 --- [o-62188-exec-12] c.n.turbine.data.AggDataFromCluster :
Fri Feb 12 22:35:52 UTC 2016 [App] STDOUT 2016-02-12 22:35:52.654 INFO 33 --- [o-62188-exec-12] c.n.t.handler.TurbineDataDispatcher : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@40a384be]
Fri Feb 12 22:36:31 UTC 2016 [App] STDOUT 2016-02-12 22:36:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:36:31 UTC 2016 [App] STDOUT 2016-02-12 22:36:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=3
Fri Feb 12 22:36:31 UTC 2016 [App] STDOUT 2016-02-12 22:36:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:36:31 UTC 2016 [App] STDOUT 2016-02-12 22:36:31.550 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:36:31 UTC 2016 [App] STDOUT 2016-02-12 22:36:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 5
Fri Feb 12 22:36:31 UTC 2016 [App] STDOUT 2016-02-12 22:36:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:36:31 UTC 2016 [App] STDOUT 2016-02-12 22:36:31.550 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:37:31 UTC 2016 [App] STDOUT 2016-02-12 22:37:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:37:31 UTC 2016 [App] STDOUT 2016-02-12 22:37:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:37:31 UTC 2016 [App] STDOUT 2016-02-12 22:37:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:37:31 UTC 2016 [App] STDOUT 2016-02-12 22:37:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:37:31 UTC 2016 [App] STDOUT 2016-02-12 22:37:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:37:31 UTC 2016 [App] STDOUT 2016-02-12 22:37:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:37:31 UTC 2016 [App] STDOUT 2016-02-12 22:37:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:38:31 UTC 2016 [App] STDOUT 2016-02-12 22:38:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:38:31 UTC 2016 [App] STDOUT 2016-02-12 22:38:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:38:31 UTC 2016 [App] STDOUT 2016-02-12 22:38:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:38:31 UTC 2016 [App] STDOUT 2016-02-12 22:38:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:38:31 UTC 2016 [App] STDOUT 2016-02-12 22:38:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:38:31 UTC 2016 [App] STDOUT 2016-02-12 22:38:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:38:31 UTC 2016 [App] STDOUT 2016-02-12 22:38:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:39:31 UTC 2016 [App] STDOUT 2016-02-12 22:39:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:39:31 UTC 2016 [App] STDOUT 2016-02-12 22:39:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:39:31 UTC 2016 [App] STDOUT 2016-02-12 22:39:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:39:31 UTC 2016 [App] STDOUT 2016-02-12 22:39:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:39:31 UTC 2016 [App] STDOUT 2016-02-12 22:39:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:39:31 UTC 2016 [App] STDOUT 2016-02-12 22:39:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:39:31 UTC 2016 [App] STDOUT 2016-02-12 22:39:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:39:31 UTC 2016 [App] STDOUT 2016-02-12 22:39:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:40:31 UTC 2016 [App] STDOUT 2016-02-12 22:40:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:40:31 UTC 2016 [App] STDOUT 2016-02-12 22:40:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:40:31 UTC 2016 [App] STDOUT 2016-02-12 22:40:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:40:31 UTC 2016 [App] STDOUT 2016-02-12 22:40:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:40:31 UTC 2016 [App] STDOUT 2016-02-12 22:40:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:40:31 UTC 2016 [App] STDOUT 2016-02-12 22:40:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:40:31 UTC 2016 [App] STDOUT 2016-02-12 22:40:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:40:32 UTC 2016 [App] STDOUT 2016-02-12 22:40:32.258 INFO 33 --- [InstanceMonitor] c.n.t.monitor.instance.InstanceMonitor : no more data from connection to device-service.mi-ae.cn
Fri Feb 12 22:40:32 UTC 2016 [App] STDOUT 2016-02-12 22:40:32.258 INFO 33 --- [InstanceMonitor] c.n.t.monitor.instance.InstanceMonitor : Re-initing host connection: device-service.mi-ae.cn DEVICE-SERVICE
Fri Feb 12 22:41:11 UTC 2016 [App] STDOUT 2016-02-12 22:41:11.072 INFO 33 --- [o-62188-exec-16] c.n.t.s.servlet.TurbineStreamServlet : FilterCriteria: []
Fri Feb 12 22:41:11 UTC 2016 [App] STDOUT 2016-02-12 22:41:11.072 INFO 33 --- [o-62188-exec-16] c.n.t.s.servlet.TurbineStreamServlet : StatsType filters: []
Fri Feb 12 22:41:11 UTC 2016 [App] STDOUT 2016-02-12 22:41:11.072 INFO 33 --- [o-62188-exec-16] c.n.t.s.TurbineStreamingConnection : Relevance metrics config: {}
Fri Feb 12 22:41:11 UTC 2016 [App] STDOUT 2016-02-12 22:41:11.072 INFO 33 --- [o-62188-exec-16] c.n.t.s.TurbineStreamingConnection : Relevance config: []
Fri Feb 12 22:41:11 UTC 2016 [App] STDOUT 2016-02-12 22:41:11.072 INFO 33 --- [o-62188-exec-16] c.n.t.monitor.cluster.ClusterMonitor : Registering event handler for cluster monitor: StreamingHandler_b182d7fb-cc48-4057-8dae-60cfd81a9265
Fri Feb 12 22:41:11 UTC 2016 [App] STDOUT 2016-02-12 22:41:11.073 INFO 33 --- [o-62188-exec-16] c.n.t.handler.TurbineDataDispatcher :
Fri Feb 12 22:41:11 UTC 2016 [App] STDOUT Just added and starting handler tuple: StreamingHandler_b182d7fb-cc48-4057-8dae-60cfd81a9265
Fri Feb 12 22:41:11 UTC 2016 [App] STDOUT 2016-02-12 22:41:11.073 INFO 33 --- [o-62188-exec-16] c.n.turbine.data.AggDataFromCluster : Per handler dispacher started for: StreamingHandler_b182d7fb-cc48-4057-8dae-60cfd81a9265
Fri Feb 12 22:41:11 UTC 2016 [App] STDOUT 2016-02-12 22:41:11.073 INFO 33 --- [o-62188-exec-16] c.n.t.monitor.cluster.ClusterMonitor : All event handlers for cluster monitor: [StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e, StaticListener_For_Aggregator, StreamingHandler_b182d7fb-cc48-4057-8dae-60cfd81a9265, StreamingHandler_9cda3aca-52d7-4594-9bcb-38538e89c46e]
Fri Feb 12 22:41:11 UTC 2016 [App] STDOUT 2016-02-12 22:41:11.073 INFO 33 --- [o-62188-exec-16] c.n.t.monitor.cluster.ClusterMonitor : Starting up the cluster monitor for AUTH-SERVICE_agg
Fri Feb 12 22:41:31 UTC 2016 [App] STDOUT 2016-02-12 22:41:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:41:31 UTC 2016 [App] STDOUT 2016-02-12 22:41:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:41:31 UTC 2016 [App] STDOUT 2016-02-12 22:41:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:41:31 UTC 2016 [App] STDOUT 2016-02-12 22:41:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:41:31 UTC 2016 [App] STDOUT 2016-02-12 22:41:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:41:31 UTC 2016 [App] STDOUT 2016-02-12 22:41:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:41:31 UTC 2016 [App] STDOUT 2016-02-12 22:41:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:41:31 UTC 2016 [App] STDOUT 2016-02-12 22:41:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:42:31 UTC 2016 [App] STDOUT 2016-02-12 22:42:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:42:31 UTC 2016 [App] STDOUT 2016-02-12 22:42:31.552 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:42:31 UTC 2016 [App] STDOUT 2016-02-12 22:42:31.552 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:42:31 UTC 2016 [App] STDOUT 2016-02-12 22:42:31.552 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:42:31 UTC 2016 [App] STDOUT 2016-02-12 22:42:31.552 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:42:31 UTC 2016 [App] STDOUT 2016-02-12 22:42:31.552 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:42:31 UTC 2016 [App] STDOUT 2016-02-12 22:42:31.552 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:42:31 UTC 2016 [App] STDOUT 2016-02-12 22:42:31.552 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:43:31 UTC 2016 [App] STDOUT 2016-02-12 22:43:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:43:31 UTC 2016 [App] STDOUT 2016-02-12 22:43:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:43:31 UTC 2016 [App] STDOUT 2016-02-12 22:43:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:43:31 UTC 2016 [App] STDOUT 2016-02-12 22:43:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:43:31 UTC 2016 [App] STDOUT 2016-02-12 22:43:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:43:31 UTC 2016 [App] STDOUT 2016-02-12 22:43:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:43:31 UTC 2016 [App] STDOUT 2016-02-12 22:43:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:43:31 UTC 2016 [App] STDOUT 2016-02-12 22:43:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:44:31 UTC 2016 [App] STDOUT 2016-02-12 22:44:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:44:31 UTC 2016 [App] STDOUT 2016-02-12 22:44:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:44:31 UTC 2016 [App] STDOUT 2016-02-12 22:44:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:44:31 UTC 2016 [App] STDOUT 2016-02-12 22:44:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:44:31 UTC 2016 [App] STDOUT 2016-02-12 22:44:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:44:31 UTC 2016 [App] STDOUT 2016-02-12 22:44:31.552 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:44:31 UTC 2016 [App] STDOUT 2016-02-12 22:44:31.560 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:45:31 UTC 2016 [App] STDOUT 2016-02-12 22:45:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:45:31 UTC 2016 [App] STDOUT 2016-02-12 22:45:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:45:31 UTC 2016 [App] STDOUT 2016-02-12 22:45:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:45:31 UTC 2016 [App] STDOUT 2016-02-12 22:45:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:45:31 UTC 2016 [App] STDOUT 2016-02-12 22:45:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:45:31 UTC 2016 [App] STDOUT 2016-02-12 22:45:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:45:31 UTC 2016 [App] STDOUT 2016-02-12 22:45:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:45:31 UTC 2016 [App] STDOUT 2016-02-12 22:45:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:46:31 UTC 2016 [App] STDOUT 2016-02-12 22:46:31.552 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:46:31 UTC 2016 [App] STDOUT 2016-02-12 22:46:31.552 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:46:31 UTC 2016 [App] STDOUT 2016-02-12 22:46:31.552 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:46:31 UTC 2016 [App] STDOUT 2016-02-12 22:46:31.552 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:46:31 UTC 2016 [App] STDOUT 2016-02-12 22:46:31.552 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:46:31 UTC 2016 [App] STDOUT 2016-02-12 22:46:31.552 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:46:31 UTC 2016 [App] STDOUT 2016-02-12 22:46:31.552 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:47:15 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:32:15 +0000] "GET /turbine.stream?cluster=AUTH-SERVICE HTTP/1.1" 200 0 25850 "-" "Apache-HttpClient/4.3.6 (java 1.5)" 10.0.4.213:13388 x_forwarded_for:"54.223.176.248" vcap_request_id:6c4fdc90-980c-40be-7dfc-df8aeb7ff3a7 response_time:900.002771426 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:44:35 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:29:35 +0000] "GET /turbine.stream?cluster=AUTH-SERVICE HTTP/1.1" 200 0 25850 "-" "Apache-HttpClient/4.3.6 (java 1.5)" 10.0.4.213:63891 x_forwarded_for:"54.223.176.248" vcap_request_id:52c5466e-3297-4b1a-5489-8d58367b5694 response_time:900.001483882 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.569 INFO 33 --- [o-62188-exec-20] c.n.t.s.servlet.TurbineStreamServlet :
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT RETURNING FROM waitOnConnection: StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.569 INFO 33 --- [ol-150-thread-1] com.netflix.turbine.utils.WorkerThread : Worker got interrupted, shutting down
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.569 INFO 33 --- [o-62188-exec-20] c.n.t.handler.TurbineDataDispatcher : Removing event handler: StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.619 INFO 33 --- [o-62188-exec-20] c.n.turbine.data.AggDataFromCluster :
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT Removing tuple for : StreamingHandler_afff8539-a2a8-4d67-a601-2458d878072e tuple running: false
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.619 INFO 33 --- [o-62188-exec-20] c.n.t.handler.TurbineDataDispatcher : tuples : [StaticListener_For_Aggregator, StreamingHandler_b182d7fb-cc48-4057-8dae-60cfd81a9265, StreamingHandler_9cda3aca-52d7-4594-9bcb-38538e89c46e]
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.619 INFO 33 --- [o-62188-exec-20] c.n.t.handler.TurbineDataDispatcher : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@3eb9fd4e]
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT RETURNING FROM waitOnConnection: StreamingHandler_9cda3aca-52d7-4594-9bcb-38538e89c46e
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.660 INFO 33 --- [o-62188-exec-21] c.n.t.s.servlet.TurbineStreamServlet :
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.660 INFO 33 --- [o-62188-exec-21] c.n.t.handler.TurbineDataDispatcher : Removing event handler: StreamingHandler_9cda3aca-52d7-4594-9bcb-38538e89c46e
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.660 INFO 33 --- [ol-151-thread-1] com.netflix.turbine.utils.WorkerThread : Worker got interrupted, shutting down
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.710 INFO 33 --- [o-62188-exec-21] c.n.turbine.data.AggDataFromCluster :
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT Removing tuple for : StreamingHandler_9cda3aca-52d7-4594-9bcb-38538e89c46e tuple running: false
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.710 INFO 33 --- [o-62188-exec-21] c.n.t.handler.TurbineDataDispatcher : tuples : [StaticListener_For_Aggregator, StreamingHandler_b182d7fb-cc48-4057-8dae-60cfd81a9265]
Fri Feb 12 22:46:36 UTC 2016 [App] STDOUT 2016-02-12 22:46:36.710 INFO 33 --- [o-62188-exec-21] c.n.t.handler.TurbineDataDispatcher : Removed handler queue tuple for handler: HandlerQueueTuple [eventHandler=com.netflix.turbine.streaming.TurbineStreamingConnection@41b407cb]
Fri Feb 12 22:47:31 UTC 2016 [App] STDOUT 2016-02-12 22:47:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:47:31 UTC 2016 [App] STDOUT 2016-02-12 22:47:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:47:31 UTC 2016 [App] STDOUT 2016-02-12 22:47:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:47:31 UTC 2016 [App] STDOUT 2016-02-12 22:47:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:47:31 UTC 2016 [App] STDOUT 2016-02-12 22:47:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:47:31 UTC 2016 [App] STDOUT 2016-02-12 22:47:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:47:31 UTC 2016 [App] STDOUT 2016-02-12 22:47:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:48:08 UTC 2016 [App] STDOUT 2016-02-12 22:48:08.333 DEBUG 33 --- [io-62188-exec-2] o.s.b.a.e.mvc.EndpointHandlerMapping : Looking up handler method for path /
Fri Feb 12 22:48:08 UTC 2016 [App] STDOUT 2016-02-12 22:48:08.333 DEBUG 33 --- [io-62188-exec-2] o.s.b.a.e.mvc.EndpointHandlerMapping : Looking up handler method for path /error
Fri Feb 12 22:48:08 UTC 2016 [App] STDOUT 2016-02-12 22:48:08.334 DEBUG 33 --- [io-62188-exec-2] o.s.b.a.e.mvc.EndpointHandlerMapping : Did not find handler method for [/error]
Fri Feb 12 22:48:49 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:48:49 +0000] "GET / HTTP/1.1" 404 0 104 "-" "Python-urllib/2.7" 10.0.4.213:15135 x_forwarded_for:"106.120.203.232" vcap_request_id:5aceedfb-f17e-41b8-4507-a33f87eab3e8 response_time:0.004907294 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:48:08 UTC 2016 [App] STDOUT 2016-02-12 22:48:08.360 DEBUG 33 --- [o-62188-exec-15] o.s.b.a.e.mvc.EndpointHandlerMapping : Looking up handler method for path /
Fri Feb 12 22:48:08 UTC 2016 [App] STDOUT 2016-02-12 22:48:08.360 DEBUG 33 --- [o-62188-exec-15] o.s.b.a.e.mvc.EndpointHandlerMapping : Did not find handler method for [/]
Fri Feb 12 22:48:08 UTC 2016 [App] STDOUT 2016-02-12 22:48:08.360 DEBUG 33 --- [o-62188-exec-15] o.s.b.a.e.mvc.EndpointHandlerMapping : Looking up handler method for path /error
Fri Feb 12 22:48:08 UTC 2016 [App] STDOUT 2016-02-12 22:48:08.360 DEBUG 33 --- [o-62188-exec-15] o.s.b.a.e.mvc.EndpointHandlerMapping : Did not find handler method for [/error]
Fri Feb 12 22:48:49 UTC 2016 [RTR] STDOUT turbine-service.mi-ae.cn - [12/02/2016:22:48:49 +0000] "GET / HTTP/1.1" 404 0 104 "-" "Python-urllib/2.7" 10.0.4.213:15135 x_forwarded_for:"106.120.203.232" vcap_request_id:151ee27a-bfd0-4587-4be7-5c42a4138262 response_time:0.004306740 app_id:871041c8-70db-4e12-ba8e-d53a5d018aae
Fri Feb 12 22:48:31 UTC 2016 [App] STDOUT 2016-02-12 22:48:31.552 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instance list for apps: [AUTH-SERVICE, DEVICE-SERVICE]
Fri Feb 12 22:48:31 UTC 2016 [App] STDOUT 2016-02-12 22:48:31.552 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:48:31 UTC 2016 [App] STDOUT 2016-02-12 22:48:31.552 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:48:31 UTC 2016 [App] STDOUT 2016-02-12 22:48:31.552 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Hosts up:2, hosts down: 0
Fri Feb 12 22:48:31 UTC 2016 [App] STDOUT 2016-02-12 22:48:31.552 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
Fri Feb 12 22:48:53 UTC 2016 [App] STDOUT 2016-02-12 22:48:53.848 INFO 33 --- [InstanceMonitor] c.n.t.monitor.instance.InstanceMonitor : Re-initing host connection: auth-service.mi-ae.cn AUTH-SERVICE
Fri Feb 12 22:49:31 UTC 2016 [App] STDOUT 2016-02-12 22:49:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: AUTH-SERVICE
Fri Feb 12 22:49:31 UTC 2016 [App] STDOUT 2016-02-12 22:49:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: AUTH-SERVICE, size=2
Fri Feb 12 22:49:31 UTC 2016 [App] STDOUT 2016-02-12 22:49:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Fetching instances for app: DEVICE-SERVICE
Fri Feb 12 22:49:31 UTC 2016 [App] STDOUT 2016-02-12 22:49:31.551 INFO 33 --- [ Timer-0] o.s.c.n.turbine.EurekaInstanceDiscovery : Received instance list for app: DEVICE-SERVICE, size=2
Fri Feb 12 22:49:31 UTC 2016 [App] STDOUT 2016-02-12 22:49:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 4
Fri Feb 12 22:49:31 UTC 2016 [App] STDOUT 2016-02-12 22:49:31.551 INFO 33 --- [ Timer-0] c.n.t.discovery.InstanceObservable : Found hosts that have been previously terminated: 0
singerdmx commented 8 years ago

There is nothing wrong with DEVICE-SERVICE, which is the service that we are trying to collect metrics. The issue is with turbine service. The same code of turbine service runs fine on local mac pro. But after pushing to cloud foundry, it stopped working. I am attaching the code here. turbine-service.zip

singerdmx commented 8 years ago

The issue seems to be solved by adding turbine.InstanceMonitor.eventStream.skipLineLogic.enabled=false Reference is http://stackoverflow.com/questions/28973994/spring-cloud-turbine-empty-stream

We are still monitoring over the weekend to see if it is truly solved

singerdmx commented 8 years ago

I believe this issue can be closed now. I do suggest that turbine logs the action of skipping or ignoring data stream. This cost us quite a few days of investigation. Thanks!