search

operator-framework / operator-lifecycle-manager

A management framework for extending Kubernetes with Operators
https://olm.operatorframework.io
Apache License 2.0
1.69k stars 543 forks source link

Flake: TestConnectionEvents deadlock? #2092

Open benluddy opened 3 years ago

benluddy commented 3 years ago

Saw this here https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/2090/pull-ci-operator-framework-operator-lifecycle-manager-master-unit/1380242533942038528#1:build-log.txt%3A48, looks like a standoff:

INFO[2021-04-08T19:50:20Z] panic: test timed out after 10m0s            
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 185 [running]:                     
INFO[2021-04-08T19:50:20Z] testing.(*M).startAlarm.func1()              
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/testing/testing.go:1701 +0x11f 
INFO[2021-04-08T19:50:20Z] created by time.goFunc                       
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/time/sleep.go:180 +0x52   
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 1 [chan receive, 10 minutes]:      
INFO[2021-04-08T19:50:20Z] testing.(*T).Run(0xc00039ec00, 0x23879d5, 0x14, 0x2434c88, 0x0) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/testing/testing.go:1240 +0x610 
INFO[2021-04-08T19:50:20Z] testing.runTests.func1(0xc00039ec00)         
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/testing/testing.go:1512 +0xa7 
INFO[2021-04-08T19:50:20Z] testing.tRunner(0xc00039ec00, 0xc0007bfce0)  
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/testing/testing.go:1194 +0x203 
INFO[2021-04-08T19:50:20Z] testing.runTests(0xc0002824c8, 0x329c9b0, 0x1, 0x1, 0xc013f57d9455a068, 0x8c300fb3ee, 0x333d600, 0xc0000ae1a8) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/testing/testing.go:1510 +0x613 
INFO[2021-04-08T19:50:20Z] testing.(*M).Run(0xc0006c3c00, 0x0)          
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/testing/testing.go:1418 +0x3b4 
INFO[2021-04-08T19:50:20Z] main.main()                                  
INFO[2021-04-08T19:50:20Z]  _testmain.go:43 +0x237                      
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 6 [chan receive]:                  
INFO[2021-04-08T19:50:20Z] k8s.io/klog/v2.(*loggingT).flushDaemon(0x333da80) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/k8s.io/klog/v2/klog.go:1169 +0xb1 
INFO[2021-04-08T19:50:20Z] created by k8s.io/klog/v2.init.0             
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/k8s.io/klog/v2/klog.go:417 +0x1a7 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 7 [chan receive]:                  
INFO[2021-04-08T19:50:20Z] k8s.io/klog.(*loggingT).flushDaemon(0x333d9a0) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/k8s.io/klog/klog.go:1010 +0xb1 
INFO[2021-04-08T19:50:20Z] created by k8s.io/klog.init.0                
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/k8s.io/klog/klog.go:411 +0x190 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 31 [IO wait, 10 minutes]:          
INFO[2021-04-08T19:50:20Z] internal/poll.runtime_pollWait(0x7f7cad2c4da0, 0x72, 0x32c3f40) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/runtime/netpoll.go:222 +0x55 
INFO[2021-04-08T19:50:20Z] internal/poll.(*pollDesc).wait(0xc0002a2198, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xe6 
INFO[2021-04-08T19:50:20Z] internal/poll.(*pollDesc).waitRead(...)      
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 
INFO[2021-04-08T19:50:20Z] internal/poll.(*FD).Read(0xc0002a2180, 0xc0002e8000, 0x8000, 0x8000, 0x0, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/internal/poll/fd_unix.go:166 +0x28f 
INFO[2021-04-08T19:50:20Z] net.(*netFD).Read(0xc0002a2180, 0xc0002e8000, 0x8000, 0x8000, 0x9, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/net/fd_posix.go:55 +0x69  
INFO[2021-04-08T19:50:20Z] net.(*conn).Read(0xc0004b2cd0, 0xc0002e8000, 0x8000, 0x8000, 0xc0002e8000, 0x9, 0x4653c0) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/net/net.go:183 +0xec      
INFO[2021-04-08T19:50:20Z] bufio.(*Reader).Read(0xc0002e24e0, 0xc0002aa038, 0x9, 0x9, 0x10401, 0x1040100000000, 0x0) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/bufio/bufio.go:227 +0x7ba 
INFO[2021-04-08T19:50:20Z] io.ReadAtLeast(0x2606c40, 0xc0002e24e0, 0xc0002aa038, 0x9, 0x9, 0x9, 0x28, 0x2257360, 0x421701) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/io/io.go:328 +0x9d        
INFO[2021-04-08T19:50:20Z] io.ReadFull(...)                             
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/io/io.go:347              
INFO[2021-04-08T19:50:20Z] golang.org/x/net/http2.readFrameHeader(0xc0002aa038, 0x9, 0x9, 0x2606c40, 0xc0002e24e0, 0x0, 0x0, 0xc000094d20, 0x418f18) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/golang.org/x/net/http2/frame.go:237 +0xb1 
INFO[2021-04-08T19:50:20Z] golang.org/x/net/http2.(*Framer).ReadFrame(0xc0002aa000, 0xc00021cb70, 0xc00021cb00, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/golang.org/x/net/http2/frame.go:492 +0xff 
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc000214c40) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x285 
INFO[2021-04-08T19:50:20Z] created by google.golang.org/grpc/internal/transport.newHTTP2Client 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0x13b7 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 53 [chan receive, 10 minutes]:     
INFO[2021-04-08T19:50:20Z] testing.(*T).Run(0xc00039ed80, 0x237372a, 0x5, 0xc000144b40, 0x594520) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/testing/testing.go:1240 +0x610 
INFO[2021-04-08T19:50:20Z] github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc.TestConnectionEvents(0xc00039ed80) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc/source_test.go:151 +0x447 
INFO[2021-04-08T19:50:20Z] testing.tRunner(0xc00039ed80, 0x2434c88)     
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/testing/testing.go:1194 +0x203 
INFO[2021-04-08T19:50:20Z] created by testing.(*T).Run                  
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/testing/testing.go:1239 +0x5d8 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 54 [chan receive, 10 minutes]:     
INFO[2021-04-08T19:50:20Z] github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc.TestConnectionEvents.func1.1(0xc00039ef00) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc/source_test.go:113 +0xa5d 
INFO[2021-04-08T19:50:20Z] testing.tRunner(0xc00039ef00, 0xc000144b40)  
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/testing/testing.go:1194 +0x203 
INFO[2021-04-08T19:50:20Z] created by testing.(*T).Run                  
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/testing/testing.go:1239 +0x5d8 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 57 [IO wait, 10 minutes]:          
INFO[2021-04-08T19:50:20Z] internal/poll.runtime_pollWait(0x7f7cad2c4e88, 0x72, 0x72) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/runtime/netpoll.go:222 +0x55 
INFO[2021-04-08T19:50:20Z] internal/poll.(*pollDesc).wait(0xc0006c3c98, 0x72, 0x0, 0x0, 0x23763b0) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xe6 
INFO[2021-04-08T19:50:20Z] internal/poll.(*pollDesc).waitRead(...)      
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 
INFO[2021-04-08T19:50:20Z] internal/poll.(*FD).Accept(0xc0006c3c80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/internal/poll/fd_unix.go:401 +0x32f 
INFO[2021-04-08T19:50:20Z] net.(*netFD).accept(0xc0006c3c80, 0x0, 0xc0002dfce0, 0x492d35) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/net/fd_unix.go:172 +0x58  
INFO[2021-04-08T19:50:20Z] net.(*TCPListener).accept(0xc000282510, 0x0, 0xc0002dfcf0, 0x4976b3) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/net/tcpsock_posix.go:139 +0x50 
INFO[2021-04-08T19:50:20Z] net.(*TCPListener).Accept(0xc000282510, 0x2435180, 0xc000186000, 0x264c5b8, 0xc0004b2cd8) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/net/tcpsock.go:261 +0x8e  
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc.(*Server).Serve(0xc000186000, 0x2639720, 0xc000282510, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/server.go:597 +0x322 
INFO[2021-04-08T19:50:20Z] github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc.server.func1() 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc/source_test.go:35 +0x58 
INFO[2021-04-08T19:50:20Z] created by github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc.TestConnectionEvents.func1.1 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc/source_test.go:93 +0x1d0 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 59 [select, 10 minutes]:           
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc00003cbc0) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0x197 
INFO[2021-04-08T19:50:20Z] created by google.golang.org/grpc.newCCBalancerWrapper 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x2fc 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 60 [chan receive, 10 minutes]:     
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc.(*addrConn).resetTransport(0xc0000ecb00) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/clientconn.go:1179 +0x8fd 
INFO[2021-04-08T19:50:20Z] created by google.golang.org/grpc.(*addrConn).connect 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/clientconn.go:815 +0x107 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 61 [select]:                       
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc.(*ClientConn).WaitForStateChange(0xc0001d1880, 0x263c228, 0xc0002e2a80, 0x2, 0xc0002e2a80) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/clientconn.go:510 +0x15e 
INFO[2021-04-08T19:50:20Z] github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc.(*SourceStore).watch.func1(0x263c1b8, 0xc00003cc80, 0xc000159a40, 0xc000097f10, 0xc0007ac390, 0xf, 0xc013f4e7949b7c26, 0x7d8c1f5c, 0x333d600, 0x0, ...) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc/source.go:147 +0x14c 
INFO[2021-04-08T19:50:20Z] github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc.(*SourceStore).watch(0xc000159a40, 0x263c1b8, 0xc00003cc80, 0x2373472, 0x4, 0x2373472, 0x4, 0xc0007ac390, 0xf, 0xc013f4e7949b7c26, ...) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc/source.go:167 +0x13a 
INFO[2021-04-08T19:50:20Z] created by github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc.(*SourceStore).Add 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc/source.go:125 +0x4cc 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 32 [select, 10 minutes]:           
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0004a0320, 0x1, 0x0, 0x0, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x1ea 
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0002e25a0, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x2df 
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc000214c40) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x12e 
INFO[2021-04-08T19:50:20Z] created by google.golang.org/grpc/internal/transport.newHTTP2Client 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0x1869 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 66 [select, 10 minutes]:           
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0004a0550, 0x1, 0x0, 0x0, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x1ea 
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0007c2000, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x2df 
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc/internal/transport.newHTTP2Server.func2(0xc0002e6300) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/http2_server.go:296 +0x1fa 
INFO[2021-04-08T19:50:20Z] created by google.golang.org/grpc/internal/transport.newHTTP2Server 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/http2_server.go:293 +0x1905 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 67 [select, 10 minutes]:           
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc0002e6300) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/http2_server.go:968 +0x412 
INFO[2021-04-08T19:50:20Z] created by google.golang.org/grpc/internal/transport.newHTTP2Server 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/http2_server.go:302 +0x192a 
INFO[2021-04-08T19:50:20Z]                                              
INFO[2021-04-08T19:50:20Z] goroutine 68 [IO wait, 10 minutes]:          
INFO[2021-04-08T19:50:20Z] internal/poll.runtime_pollWait(0x7f7cad2c4cb8, 0x72, 0x32c3f40) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/runtime/netpoll.go:222 +0x55 
INFO[2021-04-08T19:50:20Z] internal/poll.(*pollDesc).wait(0xc0002a2298, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xe6 
INFO[2021-04-08T19:50:20Z] internal/poll.(*pollDesc).waitRead(...)      
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 
INFO[2021-04-08T19:50:20Z] internal/poll.(*FD).Read(0xc0002a2280, 0xc000744000, 0x8000, 0x8000, 0x0, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/internal/poll/fd_unix.go:166 +0x28f 
INFO[2021-04-08T19:50:20Z] net.(*netFD).Read(0xc0002a2280, 0xc000744000, 0x8000, 0x8000, 0x9, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/net/fd_posix.go:55 +0x69  
INFO[2021-04-08T19:50:20Z] net.(*conn).Read(0xc0004b2cd8, 0xc000744000, 0x8000, 0x8000, 0xc000744000, 0x9, 0x4653c0) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/net/net.go:183 +0xec      
INFO[2021-04-08T19:50:20Z] bufio.(*Reader).Read(0xc0002e2600, 0xc0002aa118, 0x9, 0x9, 0x10401, 0x1040100000000, 0x0) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/bufio/bufio.go:227 +0x7ba 
INFO[2021-04-08T19:50:20Z] io.ReadAtLeast(0x2606c40, 0xc0002e2600, 0xc0002aa118, 0x9, 0x9, 0x9, 0x28, 0x2257360, 0x421701) 
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/io/io.go:328 +0x9d        
INFO[2021-04-08T19:50:20Z] io.ReadFull(...)                             
INFO[2021-04-08T19:50:20Z]  /usr/local/go/src/io/io.go:347              
INFO[2021-04-08T19:50:20Z] golang.org/x/net/http2.readFrameHeader(0xc0002aa118, 0x9, 0x9, 0x2606c40, 0xc0002e2600, 0x0, 0x0, 0xc0002d8c78, 0x418f18) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/golang.org/x/net/http2/frame.go:237 +0xb1 
INFO[2021-04-08T19:50:20Z] golang.org/x/net/http2.(*Framer).ReadFrame(0xc0002aa0e0, 0xc000666600, 0x333d600, 0x0, 0x0) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/golang.org/x/net/http2/frame.go:492 +0xff 
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc0002e6300, 0xc0006665d0, 0x24351b8) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/internal/transport/http2_server.go:458 +0x125 
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc.(*Server).serveStreams(0xc000186000, 0x264e938, 0xc0002e6300) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/server.go:718 +0x1c8 
INFO[2021-04-08T19:50:20Z] google.golang.org/grpc.(*Server).handleRawConn.func1(0xc000186000, 0x264e938, 0xc0002e6300) 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/server.go:679 +0x4d 
INFO[2021-04-08T19:50:20Z] created by google.golang.org/grpc.(*Server).handleRawConn 
INFO[2021-04-08T19:50:20Z]  /go/src/github.com/operator-framework/operator-lifecycle-manager/vendor/google.golang.org/grpc/server.go:678 +0x73e 
INFO[2021-04-08T19:50:20Z] FAIL github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc   602.131s
exdx commented 3 years ago

How consistent is this flag and which versions of OLM does it occur on?

benluddy commented 3 years ago

I've only seen this the one time, which was a run against master last week.

benluddy commented 3 years ago

Another one: https://github.com/operator-framework/operator-lifecycle-manager/pull/2137.

exdx commented 3 years ago

https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/2137/pull-ci-operator-framework-operator-lifecycle-manager-master-unit/1389594780970782720

ecordell commented 3 years ago

I think this was fixed by https://github.com/operator-framework/operator-lifecycle-manager/pull/2310, but since I couldn't reproduce we should keep this open and close only if we don't see the flake for a while.