Service Logs fails on many services

crazyxoma commented 6 years ago

[x] This is a bug report
[ ] This is a feature request
[x] I searched existing issues before opening this one

We have server with docker in swarm mode. One node. 123 containers.

I'm trying to run 3 more services.

Expected behavior

3 more services and containers running.

Actual behavior

I get the error:

error from daemon in stream: Error grabbing logs: rpc error: code = Unknown desc = warning: incomplete log stream. some logs could not be retrieved for the following reasons: task pnjhjfymyltbk64sqrfdpjly2 has not been scheduled

Steps to reproduce the behavior

Prabably - this error depends of count of services... If I'm trying to add 3 more services in other stack - everithing is ok... 3 services runs properly...

But if I use network of the 1st stack as external network - I have the same problem....

Output of docker version:

Client:
 Version:           18.06.0-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        0ffa825
 Built:             Wed Jul 18 19:09:54 2018
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.0-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       0ffa825
  Built:            Wed Jul 18 19:07:56 2018
  OS/Arch:          linux/amd64
  Experimental:     false

Output of docker info:

Containers: 123
 Running: 123
 Paused: 0
 Stopped: 0
Images: 15
Server Version: 18.06.0-ce
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: xfs
 Dirs: 3339
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
 NodeID: oiq0vttyktmleaaz5h668j4ay
 Is Manager: true
 ClusterID: 891z1rfjmtx4yd577r1z5dsdc
 Managers: 1
 Nodes: 2
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 10
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: 176.122.23.54
 Manager Addresses:
  176.122.23.54:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: d64c661f1d51c48782c9cec8fda7604785f93587
runc version: 69663f0bd4b60df09991c08812a60108003fa340
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.15.0-30-generic
Operating System: Ubuntu 18.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.41GiB
Name: ubuntu1804
ID: KQJA:SBL2:RSLQ:P4BG:KPLA:TRWX:IYSX:RTQM:CGZR:5U4G:V3LF:ZX57
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.) Ubuntu 18.04x64 Cisco UCS B480-M5 (3 GHz Skylake) x 8 DDR3 = 24GB SSD = 300GB

Top KiB Mem : 32940348 total, 839476 free, 23841232 used, 8259640 buff/cache KiB Swap: 999420 total, 999420 free, 0 used. 8098484 avail Mem

cristinagp commented 6 years ago

I get a similar error while in swarm mode, two nodes:

error from daemon in stream: Error grabbing logs: rpc error: code = Unknown desc = warning: incomplete log stream. some logs could not be retrieved for the following reasons: node xxxxxxxxxxxxxxxxxxxx is not available

crazyxoma commented 6 years ago

I tried to run the second node, but I didn’t see the default network... In my case - I have only one manager node.

crazyxoma commented 6 years ago

Is it possible to run docker swarm with IPv6 support?

docker / for-linux