search

cloudfoundry / loggregator-release

Cloud Native Logging
Apache License 2.0
215 stars 149 forks source link

[Flaky Test] Metron accepts connection on V2 API #255

Closed bradylove closed 7 years ago

bradylove commented 7 years ago

https://loggregator.ci.cf-app.com/teams/main/pipelines/loggregator/jobs/run-windows-tests/builds/295

STEP: making sure metron was build
STEP: starting metron
STEP: waiting for metron to listen
[e][metron] 2017/04/13 18:02:29 Startup: Setting up the Metron agent
[e][metron] 2017/04/13 18:02:29 metron v2 API started on addr 127.0.0.1:54010
[e][metron] 2017/04/13 18:02:29 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
[e][metron] 2017/04/13 18:02:29 Listening on 127.0.0.1:59380
[e][metron] 2017/04/13 18:02:29 metron v1 API started on addr 127.0.0.1:59380
[e][metron] 2017/04/13 18:02:29 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:29 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:29 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:29 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:29 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:29 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:29 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:29 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:29 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:29 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:29 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:29 successfully connected to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:29 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:29 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:29 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:29 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:29 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:29 successfully connected to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:29 successfully connected to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:29 successfully connected to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:29 successfully established a stream to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:29 successfully established a stream to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:29 successfully established a stream to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:29 successfully established a stream to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:29 Starting pprof server on: 127.0.0.1:54011
[e][metron] 2017/04/13 18:02:30 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:30 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:30 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:30 Failed to dial [::1]:54009: grpc: the connection is closing; please retry.
[e][metron] 2017/04/13 18:02:30 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:30 Failed to dial [::1]:54009: grpc: the connection is closing; please retry.
[e][metron] 2017/04/13 18:02:30 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:30 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:30 successfully connected to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:30 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:30 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:30 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:30 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:30 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:30 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: operation was canceled"; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:30 Failed to dial [::1]:54009: grpc: the connection is closing; please retry.
[e][metron] 2017/04/13 18:02:30 Failed to dial [::1]:54009: grpc: the connection is closing; please retry.
[e][metron] 2017/04/13 18:02:30 Failed to dial [::1]:54009: grpc: the connection is closing; please retry.
[e][metron] 2017/04/13 18:02:30 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:30 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:30 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:30 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:30 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:30 successfully connected to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:30 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:30 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:30 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:30 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:30 Failed to dial [::1]:54009: grpc: the connection is closing; please retry.
[e][metron] 2017/04/13 18:02:30 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:30 successfully connected to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:30 successfully established a stream to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:30 successfully established a stream to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:30 successfully established a stream to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:31 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:31 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:31 Failed to dial [::1]:54009: context canceled; please retry.
[e][metron] 2017/04/13 18:02:31 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:31 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:31 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:31 Failed to dial [::1]:54009: grpc: the connection is closing; please retry.
[e][metron] 2017/04/13 18:02:31 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:31 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:31 successfully connected to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:31 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:31 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:31 Failed to dial [::1]:54009: context canceled; please retry.
[e][metron] 2017/04/13 18:02:31 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:31 successfully connected to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:31 successfully established a stream to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:31 successfully established a stream to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:32 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:32 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:32 Failed to dial [::1]:54009: grpc: the connection is closing; please retry.
[e][metron] 2017/04/13 18:02:32 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:32 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:33 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:33 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:33 Failed to dial [::1]:54009: context canceled; please retry.
[e][metron] 2017/04/13 18:02:33 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:33 successfully connected to doppler [::1]:54009
[e][metron] 2017/04/13 18:02:34 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:54009: connectex: No connection could be made because the target machine actively refused it."; Reconnecting to {[::1]:54009 <nil>}
[e][metron] 2017/04/13 18:02:34 error dialing doppler: error establishing ingestor stream to [::1]:54009: rpc error: code = 14 desc = grpc: the connection is unavailable
[e][metron] 2017/04/13 18:02:34 Failed to dial [::1]:54009: grpc: the connection is closing; please retry.
[e][metron] 2017/04/13 18:02:34 Failed to lookup hostport: lookup test-availability-zone.localhost: getaddrinfow: No such host is known.
[e][metron] 2017/04/13 18:02:34 successfully connected to doppler 127.0.0.1:54009
[e][metron] 2017/04/13 18:02:34 successfully established a stream to doppler 127.0.0.1:54009
2017/04/13 18:02:39 transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:54009->127.0.0.1:54027: use of closed network connection
2017/04/13 18:02:39 transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:54009->127.0.0.1:54025: use of closed network connection
2017/04/13 18:02:39 transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:54009->127.0.0.1:54028: use of closed network connection
2017/04/13 18:02:39 transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:54009->127.0.0.1:54017: use of closed network connection
2017/04/13 18:02:39 transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:54009->127.0.0.1:54031: use of closed network connection
2017/04/13 18:02:39 transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:54009->127.0.0.1:54033: use of closed network connection
2017/04/13 18:02:39 transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:54009->127.0.0.1:54034: use of closed network connection
2017/04/13 18:02:39 transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:54009->127.0.0.1:54037: use of closed network connection
2017/04/13 18:02:39 transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:54009->127.0.0.1:54018: use of closed network connection
2017/04/13 18:02:39 transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:54009->127.0.0.1:54016: use of closed network connection
[e][metron] 2017/04/13 18:02:39 transport: http2Client.notifyError got notified that the client transport was broken EOF.
[e][metron] 2017/04/13 18:02:39 transport: http2Client.notifyError got notified that the client transport was broken EOF.
[e][metron] 2017/04/13 18:02:39 transport: http2Client.notifyError got notified that the client transport was broken EOF.
[e][metron] 2017/04/13 18:02:39 transport: http2Client.notifyError got notified that the client transport was broken EOF.
[e][metron] 2017/04/13 18:02:39 transport: http2Client.notifyError got notified that the client transport was broken EOF.
[e][metron] 2017/04/13 18:02:39 transport: http2Client.notifyError got notified that the client transport was broken EOF.
[e][metron] 2017/04/13 18:02:39 transport: http2Client.notifyError got notified that the client transport was broken EOF.
[e][metron] 2017/04/13 18:02:39 transport: http2Client.notifyError got notified that the client transport was broken EOF.
[e][metron] 2017/04/13 18:02:39 transport: http2Client.notifyError got notified that the client transport was broken EOF.
[e][metron] 2017/04/13 18:02:39 transport: http2Client.notifyError got notified that the client transport was broken EOF.
[e][metron] 2017/04/13 18:02:39 error writing to doppler: rpc error: code = 13 desc = transport is closing
[e][metron] 2017/04/13 18:02:39 error writing to doppler: rpc error: code = 13 desc = transport is closing
[e][metron] 2017/04/13 18:02:39 error writing to doppler: rpc error: code = 13 desc = transport is closing
[e][metron] 2017/04/13 18:02:39 error writing to doppler: rpc error: code = 13 desc = transport is closing
[e][metron] 2017/04/13 18:02:39 error writing to doppler: rpc error: code = 13 desc = transport is closing
[e][metron] 2017/04/13 18:02:39 error writing to doppler: rpc error: code = 13 desc = transport: tls: use of closed connection
[e][metron] 2017/04/13 18:02:39 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:54009: operation was canceled"; Reconnecting to {127.0.0.1:54009 <nil>}
[e][metron] 2017/04/13 18:02:39 grpc: addrConn.transportMonitor exits due to: grpc: the connection is closing
[e][metron] 2017/04/13 18:02:39 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:54009: operation was canceled"; Reconnecting to {127.0.0.1:54009 <nil>}
[e][metron] 2017/04/13 18:02:39 grpc: addrConn.transportMonitor exits due to: grpc: the connection is closing
[e][metron] 2017/04/13 18:02:39 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:54009: operation was canceled"; Reconnecting to {127.0.0.1:54009 <nil>}
[e][metron] 2017/04/13 18:02:39 grpc: addrConn.transportMonitor exits due to: grpc: the connection is closing
2017/04/13 18:02:39 transport: http2Client.notifyError got notified that the client transport was broken read tcp 127.0.0.1:54039->127.0.0.1:54010: wsarecv: An existing connection was forcibly closed by the remote host..
+ Failure [10.541 seconds]
Metron
/tmp/build/5bd78530/loggregator/src/metron/component_tests/metron_test.go:222
  when a consumer is accepting gRPC connections
  /tmp/build/5bd78530/loggregator/src/metron/component_tests/metron_test.go:144
    accepts connections on the v2 API [It]
    /tmp/build/5bd78530/loggregator/src/metron/component_tests/metron_test.go:143

    Expected
        <chan loggregator_v2.DopplerIngress_BatchSenderServer | len:0, cap:100>: 0xc0423a1320
    to receive something.

    /tmp/build/5bd78530/loggregator/src/metron/component_tests/metron_test.go:107
cf-gitbot commented 7 years ago

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/143679309

The labels on this github issue will be updated when the story is started.