docker / for-linux

Docker Engine for Linux
https://docs.docker.com/engine/installation/
756 stars 86 forks source link

Docker-CE cannot run because of dockered error: "Containerd did not exist sucessfully" Exist status code 2 #377

Open dabagab opened 6 years ago

dabagab commented 6 years ago

Expected behavior

When I run the sudo systemctl start docker then the docker will be start.

Actual behavior

Nothing happen. I have checked with systemctl status docker.service command with the following output.

● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
   Active: activating (start) since Mon 2018-07-30 13:17:41 CEST; 14s ago
     Docs: https://docs.docker.com
 Main PID: 3239 (dockerd)
    Tasks: 14
   Memory: 22.8M
   CGroup: /system.slice/docker.service
           └─3239 /usr/bin/dockerd

Jul 30 13:17:41 localhost.localdomain dockerd[3239]:         /go/src/github.com/containerd/containerd/server/server.go:106 +0x600
Jul 30 13:17:41 localhost.localdomain dockerd[3239]: github.com/containerd/containerd/cmd/containerd/command.App.func1(0xc420506000, 0xc420506000, 0xc4204edd07)
Jul 30 13:17:41 localhost.localdomain dockerd[3239]:         /go/src/github.com/containerd/containerd/cmd/containerd/command/main.go:132 +0x5fb
Jul 30 13:17:41 localhost.localdomain dockerd[3239]: github.com/containerd/containerd/vendor/github.com/urfave/cli.HandleAction(0x562f23b73220, 0x562f23d62e38, 0xc420506000, 0xc4204fa000, 0x0)
Jul 30 13:17:41 localhost.localdomain dockerd[3239]:         /go/src/github.com/containerd/containerd/vendor/github.com/urfave/cli/app.go:502 +0xca
Jul 30 13:17:41 localhost.localdomain dockerd[3239]: github.com/containerd/containerd/vendor/github.com/urfave/cli.(*App).Run(0xc4204a6000, 0xc42003a090, 0x3, 0x3, 0x0, 0x0)
Jul 30 13:17:41 localhost.localdomain dockerd[3239]:         /go/src/github.com/containerd/containerd/vendor/github.com/urfave/cli/app.go:268 +0x60e
Jul 30 13:17:41 localhost.localdomain dockerd[3239]: main.main()
Jul 30 13:17:41 localhost.localdomain dockerd[3239]:         github.com/containerd/containerd/cmd/containerd/main.go:28 +0x51
Jul 30 13:17:41 localhost.localdomain dockerd[3239]: time="2018-07-30T13:17:41.669205798+02:00" level=error msg="containerd did not exit successfully" error="exit status 2" module=libcontainerd

This problem occurs from Sunday.

I have tried to remove and reinstall based on documentation but I get the same error.

The real problem is that github.com/containerd/server folder does not exist on github.

Steps to reproduce the behavior

Output of docker version:

Docker version 18.06.0-ce, build 0ffa825

Output of docker info:

This is not possible because it is not running.

Additional environment details (AWS, VirtualBox, physical, etc.) Fedora 28 workstation: 4.17.9-200.fc28.x86_64

thaJeztah commented 6 years ago

Thanks for reporting; looks like there's a problem initialising a plugin in containerd; the error occurs at this step; https://github.com/containerd/containerd/blob/d64c661f1d51c48782c9cec8fda7604785f93587/server/server.go#L106-L109

result := p.Init(initContext)
if err := initialized.Add(result); err != nil {
    return nil, errors.Wrapf(err, "could not add plugin result to plugin set")
}

Perhaps you can provide a bit more information to narrow things down;

dabagab commented 6 years ago

Daemon logs error (what I found)

Jul 30 13:26:03 localhost.localdomain dockerd[3867]: time="2018-07-30T13:26:03.664153189+02:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42025b2e0, TRANSIENT_FAILURE" module=grpc
Jul 30 13:26:03 localhost.localdomain dockerd[3867]: time="2018-07-30T13:26:03.664268740+02:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42025b2e0, CONNECTING" module=grpc
Jul 30 13:26:23 localhost.localdomain dockerd[3867]: time="2018-07-30T13:26:23.664442605+02:00" level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/docker-containerd.sock 0  <nil>}. E>
Jul 30 13:26:23 localhost.localdomain dockerd[3867]: time="2018-07-30T13:26:23.664529869+02:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42025b2e0, TRANSIENT_FAILURE" module=grpc
Jul 30 13:26:23 localhost.localdomain dockerd[3867]: time="2018-07-30T13:26:23.664641847+02:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42025b2e0, CONNECTING" module=grpc
Jul 30 13:26:43 localhost.localdomain dockerd[3867]: Failed to connect to containerd: failed to dial "/var/run/docker/containerd/docker-containerd.sock": context deadline exceeded
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43.910794995+02:00" level=info msg="libcontainerd: started new docker-containerd process" pid=3968
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43.910954366+02:00" level=info msg="parsed scheme: \"unix\"" module=grpc
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43.910966945+02:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43.910999106+02:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///var/run/docker/containerd/docker-containerd.sock 0  <nil>}]" module>
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43.911010922+02:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43.911048755+02:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42003b890, CONNECTING" module=grpc
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=info msg="starting containerd" revision=d64c661f1d51c48782c9cec8fda7604785f93587 version=v1.1.1
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1>
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.aufs" error="modprobe aufs failed: "modprobe: FATAL: Module aufs not found in dir>
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.zfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.z>
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs>
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "modprobe: FATAL: Module aufs not found in directory>
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43+02:00" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs mus>
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: panic: page 28 already freed
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: goroutine 1 [running]:
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/vendor/github.com/boltdb/bolt.(*freelist).free(0xc420235da0, 0x56e, 0x7f7e4644e000)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/vendor/github.com/boltdb/bolt/freelist.go:121 +0x2ed
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/vendor/github.com/boltdb/bolt.(*node).spill(0xc4201ea540, 0xc420509198, 0x2)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/vendor/github.com/boltdb/bolt/node.go:363 +0x1fc
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/vendor/github.com/boltdb/bolt.(*Bucket).spill(0xc4200ec018, 0xb44af5, 0x55cc31464780)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/vendor/github.com/boltdb/bolt/bucket.go:570 +0x4ba
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/vendor/github.com/boltdb/bolt.(*Tx).Commit(0xc4200ec000, 0x0, 0x0)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/vendor/github.com/boltdb/bolt/tx.go:163 +0x121
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/vendor/github.com/boltdb/bolt.(*DB).Update(0xc4202081e0, 0xc420509540, 0x0, 0x0)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/vendor/github.com/boltdb/bolt/db.go:605 +0xea
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/metadata.(*DB).Init(0xc4204a63c0, 0x55cc30d6f220, 0xc4200c6010, 0xc420509660, 0xc4204a63c0)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/metadata/db.go:103 +0xb1
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/server.LoadPlugins.func2(0xc4201ea3f0, 0xc4202a9e30, 0x21, 0xc4203272a0, 0x1e)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/server/server.go:255 +0x53f
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/plugin.(*Registration).Init(0xc42026c140, 0xc4201ea3f0, 0xc42026c140)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/plugin/plugin.go:98 +0x3a
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/server.New(0x7f7e464d7b10, 0xc4200c6010, 0xc420237c20, 0x1, 0xc420509c80, 0x55cc2fabc98d)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/server/server.go:106 +0x600
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/cmd/containerd/command.App.func1(0xc4204202c0, 0xc4204202c0, 0xc4202f3d07)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/cmd/containerd/command/main.go:132 +0x5fb
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/vendor/github.com/urfave/cli.HandleAction(0x55cc30b5c220, 0x55cc30d4be38, 0xc4204202c0, 0xc42048e0c0, 0x0)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/vendor/github.com/urfave/cli/app.go:502 +0xca
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: github.com/containerd/containerd/vendor/github.com/urfave/cli.(*App).Run(0xc4201ed180, 0xc4200be060, 0x3, 0x3, 0x0, 0x0)
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         /go/src/github.com/containerd/containerd/vendor/github.com/urfave/cli/app.go:268 +0x60e
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: main.main()
Jul 30 13:26:43 localhost.localdomain dockerd[3957]:         github.com/containerd/containerd/cmd/containerd/main.go:28 +0x51
Jul 30 13:26:43 localhost.localdomain dockerd[3957]: time="2018-07-30T13:26:43.932115371+02:00" level=error msg="containerd did not exit successfully" error="exit status 2" module=libcontainerd

Updating

You mention "this problem occurs from Sunday"; was this immediately after you updated for Docker 18.06, or did you update before Sunday, and it worked before that?

When I say it "update" I mean on the sudo dnf update command only. So the previous Docker was a Docker-CE 18.06.x version too. It seems it was docker-ce-18.06.0.ce-3.fc28.x86_64. So this was that docker version when from the problem occured.

I use past tense because I could remove fully the docker-ce, restart computer, reinstall and it works now.

Transaction performed with:
    Installed     dnf-2.7.5-12.fc28.noarch @anaconda
    Installed     rpm-4.14.1-9.fc28.x86_64 @updates
Packages Altered:
    Erase checkpolicy-2.8-1.fc28.x86_64                     @updates
    Erase container-selinux-2:2.68-1.git25277c8.fc28.noarch @updates
    Erase docker-ce-18.06.0.ce-3.fc28.x86_64                @docker-ce-stable
    Erase policycoreutils-python-utils-2.8-1.fc28.noarch    @updates
    Erase python3-IPy-0.81-21.fc28.noarch                   @fedora
    Erase python3-audit-2.8.4-2.fc28.x86_64                 @updates
    Erase python3-libsemanage-2.8-2.fc28.x86_64             @updates
    Erase python3-policycoreutils-2.8-1.fc28.noarch         @updates
    Erase python3-setools-4.1.1-8.1.fc28.x86_64             @updates

Was this an upgrade from a previous version of Docker, or a fresh install? If you were running an older version of Docker before this; do you know which version? Also; were both versions installed from the official packages (https://docs.docker.com/install/linux/docker-ce/fedora/) or have you ran packages provided by the Fedora packagers before?

It was a fresh Docker-CE install on a fresh Fedora 28 (no upgrade from 27!), if I remember well there was one Docker-CE update (maybe once I see when I run the dnf update command in the past.)

Custom configuration

do you have any custom configuration options set for dockerd? (either through the systemd unit file, environment variables, or /etc/docker/daemon.json)?

No, nothing, I just followed the description from https://docs.docker.com/install/linux/docker-ce/fedora/ when I installed in spring (when Fedora 28 came out)

Script running

could you try running the check-config.sh script (can be found here;

I am so sorry, but I could fix this with full removing:

sudo dnf remove docker-ce
sudo dnf remove docker*
sudo dnf autoremove
sudo rm -rf /var/lib/docker

sudo dnf remove docker*and sudo dnf autoremove were an extra command now this time and those removed some kind of lib folder and some other python3 libraries, container-selinux too. Restart and then following the installation instructions on the official docker site.

Updating history

dnf update history on 29th of July, but I do not see nothing library which is related to docker...

Transaction performed with:
    Installed     dnf-2.7.5-12.fc28.noarch @anaconda
    Installed     rpm-4.14.1-9.fc28.x86_64 @updates
Packages Altered:
    Upgraded autocorr-en-1:6.0.5.2-1.fc28.noarch                        @updates
    Upgrade              1:6.0.6.1-5.fc28.noarch                        @updates
    Upgraded cups-1:2.2.6-15.fc28.x86_64                                @updates
    Upgrade       1:2.2.6-16.fc28.x86_64                                @updates
    Upgraded cups-client-1:2.2.6-15.fc28.x86_64                         @updates
    Upgrade              1:2.2.6-16.fc28.x86_64                         @updates
    Upgraded cups-filesystem-1:2.2.6-15.fc28.noarch                     @updates
    Upgrade                  1:2.2.6-16.fc28.noarch                     @updates
    Upgraded cups-libs-1:2.2.6-15.fc28.x86_64                           @updates
    Upgrade            1:2.2.6-16.fc28.x86_64                           @updates
    Upgraded dracut-048-6.git20180718.fc28.1.x86_64                     @updates
    Upgrade         048-14.git20180726.fc28.x86_64                      @updates
    Upgraded dracut-config-rescue-048-6.git20180718.fc28.1.x86_64       @updates
    Upgrade                       048-14.git20180726.fc28.x86_64        @updates
    Upgraded dracut-live-048-6.git20180718.fc28.1.x86_64                @updates
    Upgrade              048-14.git20180726.fc28.x86_64                 @updates
    Upgraded dracut-network-048-6.git20180718.fc28.1.x86_64             @updates
    Upgrade                 048-14.git20180726.fc28.x86_64              @updates
    Upgraded file-roller-3.28.0-1.fc28.x86_64                           @anaconda
    Upgrade              3.28.1-1.fc28.x86_64                           @updates
    Upgraded file-roller-nautilus-3.28.0-1.fc28.x86_64                  @anaconda
    Upgrade                       3.28.1-1.fc28.x86_64                  @updates
    Upgraded fpaste-0.3.9.1-3.fc28.noarch                               @anaconda
    Upgrade         0.3.9.2-1.fc28.noarch                               @updates
    Upgraded ibus-1.5.18-5.fc28.x86_64                                  @updates
    Upgrade       1.5.18-7.fc28.x86_64                                  @updates
    Upgraded ibus-gtk2-1.5.18-5.fc28.x86_64                             @updates
    Upgrade            1.5.18-7.fc28.x86_64                             @updates
    Upgraded ibus-gtk3-1.5.18-5.fc28.x86_64                             @updates
    Upgrade            1.5.18-7.fc28.x86_64                             @updates
    Upgraded ibus-libs-1.5.18-5.fc28.x86_64                             @updates
    Upgrade            1.5.18-7.fc28.x86_64                             @updates
    Upgraded ibus-setup-1.5.18-5.fc28.noarch                            @updates
    Upgrade             1.5.18-7.fc28.noarch                            @updates
    Upgraded java-1.8.0-openjdk-headless-1:1.8.0.172-12.b11.fc28.x86_64 @updates
    Upgrade                              1:1.8.0.181-7.b13.fc28.x86_64  @updates
    Upgraded libinput-1.11.2-2.fc28.x86_64                              @updates
    Upgrade           1.11.3-1.fc28.x86_64                              @updates
    Upgraded libreoffice-calc-1:6.0.5.2-1.fc28.x86_64                   @updates
    Upgrade                   1:6.0.6.1-5.fc28.x86_64                   @updates
    Upgraded libreoffice-core-1:6.0.5.2-1.fc28.x86_64                   @updates
    Upgrade                   1:6.0.6.1-5.fc28.x86_64                   @updates
    Upgraded libreoffice-data-1:6.0.5.2-1.fc28.noarch                   @updates
    Upgrade                   1:6.0.6.1-5.fc28.noarch                   @updates
    Upgraded libreoffice-draw-1:6.0.5.2-1.fc28.x86_64                   @updates
    Upgrade                   1:6.0.6.1-5.fc28.x86_64                   @updates
    Upgraded libreoffice-emailmerge-1:6.0.5.2-1.fc28.x86_64             @updates
    Upgrade                         1:6.0.6.1-5.fc28.x86_64             @updates
    Upgraded libreoffice-filters-1:6.0.5.2-1.fc28.x86_64                @updates
    Upgrade                      1:6.0.6.1-5.fc28.x86_64                @updates
    Upgraded libreoffice-graphicfilter-1:6.0.5.2-1.fc28.x86_64          @updates
    Upgrade                            1:6.0.6.1-5.fc28.x86_64          @updates
    Upgraded libreoffice-gtk2-1:6.0.5.2-1.fc28.x86_64                   @updates
    Upgrade                   1:6.0.6.1-5.fc28.x86_64                   @updates
    Upgraded libreoffice-gtk3-1:6.0.5.2-1.fc28.x86_64                   @updates
    Upgrade                   1:6.0.6.1-5.fc28.x86_64                   @updates
    Upgraded libreoffice-help-en-1:6.0.5.2-1.fc28.x86_64                @updates
    Upgrade                      1:6.0.6.1-5.fc28.x86_64                @updates
    Upgraded libreoffice-impress-1:6.0.5.2-1.fc28.x86_64                @updates
    Upgrade                      1:6.0.6.1-5.fc28.x86_64                @updates
    Upgraded libreoffice-langpack-en-1:6.0.5.2-1.fc28.x86_64            @updates
    Upgrade                          1:6.0.6.1-5.fc28.x86_64            @updates
    Upgraded libreoffice-math-1:6.0.5.2-1.fc28.x86_64                   @updates
    Upgrade                   1:6.0.6.1-5.fc28.x86_64                   @updates
    Upgraded libreoffice-opensymbol-fonts-1:6.0.5.2-1.fc28.noarch       @updates
    Upgrade                               1:6.0.6.1-5.fc28.noarch       @updates
    Upgraded libreoffice-pdfimport-1:6.0.5.2-1.fc28.x86_64              @updates
    Upgrade                        1:6.0.6.1-5.fc28.x86_64              @updates
    Upgraded libreoffice-pyuno-1:6.0.5.2-1.fc28.x86_64                  @updates
    Upgrade                    1:6.0.6.1-5.fc28.x86_64                  @updates
    Upgraded libreoffice-ure-1:6.0.5.2-1.fc28.x86_64                    @updates
    Upgrade                  1:6.0.6.1-5.fc28.x86_64                    @updates
    Upgraded libreoffice-ure-common-1:6.0.5.2-1.fc28.noarch             @updates
    Upgrade                         1:6.0.6.1-5.fc28.noarch             @updates
    Upgraded libreoffice-writer-1:6.0.5.2-1.fc28.x86_64                 @updates
    Upgrade                     1:6.0.6.1-5.fc28.x86_64                 @updates
    Upgraded libreoffice-x11-1:6.0.5.2-1.fc28.x86_64                    @updates
    Upgrade                  1:6.0.6.1-5.fc28.x86_64                    @updates
    Upgraded libreoffice-xsltfilter-1:6.0.5.2-1.fc28.x86_64             @updates
    Upgrade                         1:6.0.6.1-5.fc28.x86_64             @updates
    Upgraded libreofficekit-1:6.0.5.2-1.fc28.x86_64                     @updates
    Upgrade                 1:6.0.6.1-5.fc28.x86_64                     @updates
    Upgraded python2-setuptools-39.2.0-1.fc28.noarch                    @updates
    Upgrade                     39.2.0-6.fc28.noarch                    @updates
    Upgraded python3-setuptools-39.2.0-1.fc28.noarch                    @updates
    Upgrade                     39.2.0-6.fc28.noarch                    @updates
    Upgraded selinux-policy-3.14.1-32.fc28.noarch                       @updates
    Upgrade                 3.14.1-36.fc28.noarch                       @updates
    Upgraded selinux-policy-targeted-3.14.1-32.fc28.noarch              @updates
    Upgrade                          3.14.1-36.fc28.noarch              @updates
    Upgraded vim-minimal-2:8.1.197-1.fc28.x86_64                        @updates
    Upgrade              2:8.1.209-1.fc28.x86_64                        @updates

Maybe the selinux-policy?

history of 28th of July, because I did not coding on Saturday.

Transaction performed with:
    Installed     dnf-2.7.5-12.fc28.noarch @anaconda
    Installed     rpm-4.14.1-9.fc28.x86_64 @updates
Packages Altered:
    Upgraded avahi-0.7-12.fc28.x86_64                                  @anaconda
    Upgrade        0.7-13.fc28.x86_64                                  @updates
    Upgraded avahi-glib-0.7-12.fc28.x86_64                             @anaconda
    Upgrade             0.7-13.fc28.x86_64                             @updates
    Upgraded avahi-libs-0.7-12.fc28.x86_64                             @anaconda
    Upgrade             0.7-13.fc28.x86_64                             @updates
    Upgraded container-selinux-2:2.65-1.gitbf5b26b.fc28.noarch         (unknown)
    Upgrade                    2:2.68-1.git25277c8.fc28.noarch         @updates
    Upgraded gnutls-3.6.2-3.fc28.x86_64                                @updates
    Upgrade         3.6.3-2.fc28.x86_64                                @updates
    Upgraded gnutls-dane-3.6.2-3.fc28.x86_64                           @updates
    Upgrade              3.6.3-2.fc28.x86_64                           @updates
    Upgraded gnutls-utils-3.6.2-3.fc28.x86_64                          @updates
    Upgrade               3.6.3-2.fc28.x86_64                          @updates
    Erase    kernel-4.17.5-200.fc28.x86_64                             @updates
    Install  kernel-4.17.9-200.fc28.x86_64                             @updates
    Erase    kernel-core-4.17.5-200.fc28.x86_64                        @updates
    Install  kernel-core-4.17.9-200.fc28.x86_64                        @updates
    Erase    kernel-devel-4.17.5-200.fc28.x86_64                       @updates
    Install  kernel-devel-4.17.9-200.fc28.x86_64                       @updates
    Upgraded kernel-headers-4.17.7-200.fc28.x86_64                     @updates
    Upgrade                 4.17.9-200.fc28.x86_64                     @updates
    Erase    kernel-modules-4.17.5-200.fc28.x86_64                     @updates
    Install  kernel-modules-4.17.9-200.fc28.x86_64                     @updates
    Erase    kernel-modules-extra-4.17.5-200.fc28.x86_64               @updates
    Install  kernel-modules-extra-4.17.9-200.fc28.x86_64               @updates
    Erase    kmod-nvidia-4.17.5-200.fc28.x86_64-3:390.48-1.fc28.x86_64 @@commandline
    Upgraded mutter-3.28.3-1.fc28.x86_64                               @updates
    Upgrade         3.28.3-2.fc28.x86_64                               @updates
    Upgraded qgnomeplatform-0.3-9.fc28.x86_64                          @anaconda
    Upgrade                 0.4-1.fc28.x86_64                          @updates

maybe kernel updates caused this problem?

thaJeztah commented 6 years ago

Thanks for the additional information; looks like something happening in boltdb; I'll try to reach out to the containerd maintainers; perhaps they've seen this before

tomtom30 commented 6 years ago

Hi, I have exactly the same issue on ubuntu. Do you have resolve this? Thanks Thomas

MarkMekhaiel commented 6 years ago

Same issue - Ubuntu 18.04

jundl77 commented 6 years ago

Same as well, also Ubuntu 18.04

flik commented 6 years ago

Same issue - Ubuntu 18.04 I could not found any solution still now.

offSwitch commented 5 years ago

same issue Ubunt 18.04 Docker-ce 18.06.1 the virtual machine crashes because the disk suddenly unavailable. systemctl start docker not work after restart host.

Dec 21 16:44:46 dockerfour kubelet[2832]: W1221 16:44:46.104884    2832 manager.go:253] Timeout trying to communicate with docker during initialization, will retry
root@dockerfour:/var/run# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: activating (start) since Fri 2018-12-21 16:44:55 UTC; 9s ago
     Docs: https://docs.docker.com
 Main PID: 2893 (dockerd)
    Tasks: 9
   CGroup: /system.slice/docker.service
           └─2893 /usr/bin/dockerd -H fd://

Dec 21 16:44:56 dockerfour dockerd[2893]:         /go/src/github.com/containerd/containerd/server/server.go:106 +0x600
Dec 21 16:44:56 dockerfour dockerd[2893]: github.com/containerd/containerd/cmd/containerd/command.App.func1(0xc420464000, 0xc420464000, 0xc4202c9d07)
Dec 21 16:44:56 dockerfour dockerd[2893]:         /go/src/github.com/containerd/containerd/cmd/containerd/command/main.go:132 +0x5fb
Dec 21 16:44:56 dockerfour dockerd[2893]: github.com/containerd/containerd/vendor/github.com/urfave/cli.HandleAction(0x55b15f56a760, 0x55b15f75a718, 0xc420464000, 0xc4201a0420, 0x0)
Dec 21 16:44:56 dockerfour dockerd[2893]:         /go/src/github.com/containerd/containerd/vendor/github.com/urfave/cli/app.go:502 +0xca
Dec 21 16:44:56 dockerfour dockerd[2893]: github.com/containerd/containerd/vendor/github.com/urfave/cli.(*App).Run(0xc42045a000, 0xc42003a090, 0x3, 0x3, 0x0, 0x0)
Dec 21 16:44:56 dockerfour dockerd[2893]:         /go/src/github.com/containerd/containerd/vendor/github.com/urfave/cli/app.go:268 +0x60e
Dec 21 16:44:56 dockerfour dockerd[2893]: main.main()
Dec 21 16:44:56 dockerfour dockerd[2893]:         github.com/containerd/containerd/cmd/containerd/main.go:28 +0x51
Dec 21 16:44:56 dockerfour dockerd[2893]: time="2018-12-21T16:44:56.072795218Z" level=error msg="containerd did not exit successfully" error="exit status 2" module=libcontainerd
root@dockerfour:/var/run# journalctl -xe
Dec 21 16:47:57 dockerfour dockerd[3206]: time="2018-12-21T16:47:57Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
Dec 21 16:47:57 dockerfour dockerd[3206]: time="2018-12-21T16:47:57Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.zfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs file
Dec 21 16:47:57 dockerfour dockerd[3206]: time="2018-12-21T16:47:57Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1
Dec 21 16:47:57 dockerfour dockerd[3206]: time="2018-12-21T16:47:57Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs file
Dec 21 16:47:57 dockerfour dockerd[3206]: time="2018-12-21T16:47:57Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem
Dec 21 16:47:57 dockerfour dockerd[3206]: panic: page 30 already freed
Dec 21 16:47:57 dockerfour dockerd[3206]: goroutine 1 [running]:
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/vendor/github.com/boltdb/bolt.(*freelist).free(0xc42021d890, 0x2322, 0x7f1a01dfc000)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/vendor/github.com/boltdb/bolt/freelist.go:121 +0x2ed
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/vendor/github.com/boltdb/bolt.(*node).spill(0xc4200cc310, 0xc4204db198, 0x2)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/vendor/github.com/boltdb/bolt/node.go:363 +0x1fc
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/vendor/github.com/boltdb/bolt.(*Bucket).spill(0xc4200d6018, 0x178064a, 0x5568ae9f8780)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/vendor/github.com/boltdb/bolt/bucket.go:570 +0x4ba
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/vendor/github.com/boltdb/bolt.(*Tx).Commit(0xc4200d6000, 0x0, 0x0)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/vendor/github.com/boltdb/bolt/tx.go:163 +0x121
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/vendor/github.com/boltdb/bolt.(*DB).Update(0xc4201f21e0, 0xc4204db540, 0x0, 0x0)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/vendor/github.com/boltdb/bolt/db.go:605 +0xea
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/metadata.(*DB).Init(0xc4200ca180, 0x5568ae302b40, 0xc4200b0010, 0xc4204db660, 0xc4200ca180)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/metadata/db.go:103 +0xb1
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/server.LoadPlugins.func2(0xc4200cc230, 0xc4202fbbf0, 0x21, 0xc4200b3680, 0x1e)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/server/server.go:255 +0x53f
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/plugin.(*Registration).Init(0xc42046e4b0, 0xc4200cc230, 0xc42046e4b0)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/plugin/plugin.go:98 +0x3a
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/server.New(0x7f1a01ec3790, 0xc4200b0010, 0xc42021f9e0, 0x1, 0xc4202dbc80, 0x5568ad04e75d)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/server/server.go:106 +0x600
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/cmd/containerd/command.App.func1(0xc4204142c0, 0xc4204142c0, 0xc4202dbd07)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/cmd/containerd/command/main.go:132 +0x5fb
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/vendor/github.com/urfave/cli.HandleAction(0x5568ae0ef760, 0x5568ae2df718, 0xc4204142c0, 0xc42046c060, 0x0)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/vendor/github.com/urfave/cli/app.go:502 +0xca
Dec 21 16:47:57 dockerfour dockerd[3206]: github.com/containerd/containerd/vendor/github.com/urfave/cli.(*App).Run(0xc4201d5180, 0xc4200a8060, 0x3, 0x3, 0x0, 0x0)
Dec 21 16:47:57 dockerfour dockerd[3206]:         /go/src/github.com/containerd/containerd/vendor/github.com/urfave/cli/app.go:268 +0x60e
Dec 21 16:47:57 dockerfour dockerd[3206]: main.main()
Dec 21 16:47:57 dockerfour dockerd[3206]:         github.com/containerd/containerd/cmd/containerd/main.go:28 +0x51
Dec 21 16:47:57 dockerfour dockerd[3206]: time="2018-12-21T16:47:57.097079362Z" level=error msg="containerd did not exit successfully" error="exit status 2" module=libcontainerd
puzzledz commented 5 years ago
May 07 14:57:06 node1 systemd[1]: Starting Docker Application Container Engine...
May 07 14:57:20 node1 systemd[1]: Stopped Docker Application Container Engine.
May 07 15:19:00 node1 systemd[1]: Starting Docker Application Container Engine...
May 07 15:19:01 node1 systemd[1]: Stopped Docker Application Container Engine.
May 07 15:19:07 node1 systemd[1]: Starting Docker Application Container Engine...
May 07 15:19:08 node1 systemd[1]: Stopped Docker Application Container Engine.
May 07 15:42:39 node1 systemd[1]: Starting Docker Application Container Engine...
May 07 15:42:39 node1 dockerd[12639]: time="2019-05-07T15:42:39.474362385+08:00" level=info msg="libcontainerd: started new containerd process" pi
May 07 15:42:39 node1 dockerd[12639]: time="2019-05-07T15:42:39.480557749+08:00" level=info msg="parsed scheme: \"unix\"" module=grpc
May 07 15:42:39 node1 dockerd[12639]: time="2019-05-07T15:42:39.480673324+08:00" level=info msg="scheme \"unix\" not registered, fallback to defau
May 07 15:42:39 node1 dockerd[12639]: time="2019-05-07T15:42:39.480795328+08:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [
May 07 15:42:39 node1 dockerd[12639]: time="2019-05-07T15:42:39.480908947+08:00" level=info msg="ClientConn switching balancer to \"pick_first\""
May 07 15:42:39 node1 dockerd[12639]: time="2019-05-07T15:42:39.481068614+08:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.401733298+08:00" level=info msg="starting containerd" revision=e6b3f5632f50dbc4e9c
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.403859700+08:00" level=info msg="loading plugin "io.containerd.content.v1.content"
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.406277697+08:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrf
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.406970100+08:00" level=warning msg="failed to load plugin io.containerd.snapshotte
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.407079666+08:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.461405340+08:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.nati
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.469792511+08:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.over
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.470578898+08:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.471153140+08:00" level=warning msg="failed to load plugin io.containerd.snapshotte
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.471265431+08:00" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.483282461+08:00" level=warning msg="could not use snapshotter zfs in metadata plug
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.483794095+08:00" level=warning msg="could not use snapshotter btrfs in metadata pl
May 07 15:42:42 node1 systemd[1]: Stopped Docker Application Container Engine.
puzzledz commented 5 years ago
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Tue 2019-05-07 15:42:42 CST; 27min ago
     Docs: https://docs.docker.com
 Main PID: 12639 (code=killed, signal=TERM)

May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.406970100+08:00" level=warning msg="failed to load plugin io.containerd.snapshotte
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.407079666+08:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.461405340+08:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.nati
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.469792511+08:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.over
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.470578898+08:00" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.471153140+08:00" level=warning msg="failed to load plugin io.containerd.snapshotte
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.471265431+08:00" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.483282461+08:00" level=warning msg="could not use snapshotter zfs in metadata plug
May 07 15:42:42 node1 dockerd[12639]: time="2019-05-07T15:42:42.483794095+08:00" level=warning msg="could not use snapshotter btrfs in metadata pl
May 07 15:42:42 node1 systemd[1]: Stopped Docker Application Container Engine.

one day,the machine shutdown by accident,then second day,docker didn't work.cannot connect to the docker daemon at unix:///var/run/docker.sock.how can i restart docker.socket?docker-ce version:18.09 and virtual evironment(ubuntu 18.04),and i remove docker-ce then reinstall docker cannot solve the problem

zooqkl commented 5 years ago

This is due to the version of dockerclient being too low. After uninstalling docker, install the new version of dockerclient.

$yum list installed | grep docker $yum -y remove docker-ce.x86_64 $cd /usr/local/bin $sudo rm -rf docker docker-containerd docker-containerd-ctr docker-containerd-shim dockerd docker-proxy docker-runc $rm -rf /var/lib/docker $sudo yum install docker-ce-18.06.3.ce -y

slove the problem.