search

docker-archive / for-azure

27 stars 18 forks source link

panic: runtime error in Swarm manager node #42

Open anushkaee opened 7 years ago

anushkaee commented 7 years ago

Actual behavior

We deployed docker-ce for azure using below template: https://store.docker.com/editions/community/docker-ce-azure

but after few days , docker service on manager node got crashed.

Information

Cannot execute any docker command on manager node,

swarm-manager000000:~$ docker-diagnose Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Following error has appeared at the end of the docker.log file.

Stack trace:

Oct 26 04:41:43 moby root: time="2017-10-26T04:41:43.239733160Z" level=debug msg=subscribed method="(*LogBroker).SubscribeLogs" subscription.id=v61j7ly06ey19w6gvscclp6ri  
Oct 26 04:41:43 moby root: panic: runtime error: index out of range 
Oct 26 04:41:43 moby root: goroutine 1174563 [running]: 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/github.com/docker/swarmkit/api.(*SubscriptionMessage).MarshalTo(0xc423f6a630, 0xc4224a67d0, 0x47, 0x47, 0x47, 0x47, 0x1a4d120) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/github.com/docker/swarmkit/api/logbroker.pb.go:1162 +0x34a 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/github.com/docker/swarmkit/api.(*SubscriptionMessage).Marshal(0xc423f6a630, 0x7fd07c460088, 0xc423f6a630, 0x7fd07c4600c0, 0xc423f6a630, 0xe729201) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/github.com/docker/swarmkit/api/logbroker.pb.go:1123 +0x84 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/github.com/golang/protobuf/proto.(*Buffer).Marshal(0xc421e8e0d8, 0x7fd07c460088, 0xc423f6a630, 0xc4230b83c0, 0x0) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/github.com/golang/protobuf/proto/encode.go:264 +0x7a 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/google.golang.org/grpc.protoCodec.marshal(0x1a4d120, 0xc423f6a630, 0xc421e8e0d0, 0x43efe5, 0xc42642f790, 0x3, 0x3, 0xc42642f820) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/google.golang.org/grpc/codec.go:78 +0xe8 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/google.golang.org/grpc.protoCodec.Marshal(0x1a4d120, 0xc423f6a630, 0x0, 0x3, 0x3, 0x3, 0x0) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/google.golang.org/grpc/codec.go:88 +0x73 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/google.golang.org/grpc.(*protoCodec).Marshal(0x28aa198, 0x1a4d120, 0xc423f6a630, 0xc425860008, 0xc8, 0xc8, 0xc42642f498, 0x40d219) 
Oct 26 04:41:43 moby root: ^I<autogenerated>:35 +0x59 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/google.golang.org/grpc.encode(0x2832620, 0x28aa198, 0x1a4d120, 0xc423f6a630, 0x0, 0x0, 0x0, 0x0, 0xc425f46920, 0xc425f468b8, ...) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/google.golang.org/grpc/rpc_util.go:253 +0x2f9 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/google.golang.org/grpc.(*serverStream).SendMsg(0xc4259b0c80, 0x1a4d120, 0xc423f6a630, 0x0, 0x0) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/google.golang.org/grpc/stream.go:581 +0x113 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/github.com/grpc-ecosystem/go-grpc-prometheus.(*monitoredServerStream).SendMsg(0xc424b31ce0, 0x1a4d120, 0xc423f6a630, 0x377dc7e7aad672ec, 0xc4244b54d0) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/github.com/grpc-ecosystem/go-grpc-prometheus/server.go:61 +0x4b 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/github.com/docker/swarmkit/api.(*logBrokerListenSubscriptionsServer).Send(0xc425804b50, 0xc423f6a630, 0xc425860340, 0xc42237b080) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/github.com/docker/swarmkit/api/logbroker.pb.go:748 +0x49 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/github.com/docker/swarmkit/api.(*LogBroker_ListenSubscriptionsServerWrapper).Send(0xc424b31d20, 0xc423f6a630, 0xc423f6a630, 0xc425f472e0) 
Oct 26 04:41:43 moby root: ^I<autogenerated>:459 +0x53 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/github.com/docker/swarmkit/manager/logbroker.(*LogBroker).ListenSubscriptions(0xc421695b00, 0x28aa198, 0x283a600, 0xc424b31d20, 0x0, 0x0) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/github.com/docker/swarmkit/manager/logbroker/broker.go:368 +0xa8d 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/github.com/docker/swarmkit/api.(*authenticatedWrapperLogBrokerServer).ListenSubscriptions(0xc420e14780, 0x28aa198, 0x283a600, 0xc424b31d20, 0x0, 0x0) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/github.com/docker/swarmkit/api/logbroker.pb.go:276 +0x127 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/github.com/docker/swarmkit/api.(*raftProxyLogBrokerServer).ListenSubscriptions(0xc42093bb80, 0x28aa198, 0x2839fa0, 0xc425804b50, 0xc42093bb80, 0x4120b8) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/github.com/docker/swarmkit/api/logbroker.pb.go:1483 +0x23e 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/github.com/docker/swarmkit/api._LogBroker_ListenSubscriptions_Handler(0x1962e80, 0xc42093bb80, 0x28384a0, 0xc424b31ce0, 0xc4228ae280, 0xc421553c00) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/github.com/docker/swarmkit/api/logbroker.pb.go:735 +0x113 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/github.com/grpc-ecosystem/go-grpc-prometheus.StreamServerInterceptor(0x1962e80, 0xc42093bb80, 0x2838740, 0xc4259b0c80, 0xc424b31cc0, 0x1be9f18, 0xffffffffffffffff, 0xc4207f06c8) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/github.com/grpc-ecosystem/go-grpc-prometheus/server.go:40 +0x13b 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/google.golang.org/grpc.(*Server).processStreamingRPC(0xc4210a10e0, 0x283a240, 0xc42134b1e0, 0xc424855680, 0xc421f381e0, 0x27f5b60, 0xc42589bc20, 0x0, 0x0) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/google.golang.org/grpc/server.go:872 +0x363 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/google.golang.org/grpc.(*Server).handleStream(0xc4210a10e0, 0x283a240, 0xc42134b1e0, 0xc424855680, 0xc42589bc20) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/google.golang.org/grpc/server.go:959 +0x1539 
Oct 26 04:41:43 moby root: github.com/docker/docker/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc4254c1880, 0xc4210a10e0, 0x283a240, 0xc42134b1e0, 0xc424855680) 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/google.golang.org/grpc/server.go:517 +0xa9 
Oct 26 04:41:43 moby root: created by github.com/docker/docker/vendor/google.golang.org/grpc.(*Server).serveStreams.func1 
Oct 26 04:41:43 moby root: ^I/go/src/github.com/docker/docker/vendor/google.golang.org/grpc/server.go:518 +0xa1
FrenchBen commented 7 years ago

What were you running on the cluster? Can you obtain a stack trace and attach it here? https://docs.docker.com/engine/admin/#force-a-stack-trace-to-be-logged