search

elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
12.15k stars 4.91k forks source link

[Dockerlogbeat] beatSocket.sock: connect: no such file or directory #29468

Closed gergof closed 10 months ago

gergof commented 2 years ago

After installing the plugin it can't be enabled. The output of the install command:

$ docker plugin install elastic/elastic-logging-plugin:7.16.1
Plugin "elastic/elastic-logging-plugin:7.16.1" is requesting the following privileges:
 - network: [host]
 - mount: [/var/lib/docker]
Do you grant the above permissions? [y/N] y
7.16.1: Pulling from elastic/elastic-logging-plugin
Digest: sha256:88e8ec3a7c6b9191301243345c1fe13020892fbaa35c5e85d8388a4bcb05f7e1
8cefd11609cf: Complete 
Error response from daemon: dial unix /run/docker/plugins/62d7ef0a88a0ad1e22f74da6241c3c7a636f9344cdbcb3b31bd3df97d36ab8e1/beatSocket.sock: connect: no such file or directory

I've tried to enable debugging with docker plugin set elastic/elastic-logging-plugin:7.16.1 LOG_DRIVER_LEVEL=debug and here are the journald entries after trying to enable the plugin:

dec 16 12:00:13 glap2 dockerd[15689]: time="2021-12-16T12:00:13+02:00" level=error msg="standard_init_linux.go:228: exec user process caused: no such file or directory" plugin=62d7ef0a88a0ad1e22f74da6241c3c7a636f9344cdbcb3b31bd3df97d36ab8e1
dec 16 12:00:13 glap2 dockerd[15689]: time="2021-12-16T12:00:13.820334648+02:00" level=info msg="ignoring event" container=62d7ef0a88a0ad1e22f74da6241c3c7a636f9344cdbcb3b31bd3df97d36ab8e1 module=libcontainerd namespace=plugins.moby topic=/tasks/delete type="*events.TaskDelete"
dec 16 12:00:14 glap2 dockerd[15689]: time="2021-12-16T12:00:14+02:00" level=error msg="standard_init_linux.go:228: exec user process caused: no such file or directory" plugin=62d7ef0a88a0ad1e22f74da6241c3c7a636f9344cdbcb3b31bd3df97d36ab8e1
dec 16 12:00:26 glap2 dockerd[15689]: time="2021-12-16T12:00:26.343791681+02:00" level=error msg="Sending SIGTERM to plugin failed with error: process already finished: not found"
dec 16 12:00:26 glap2 dockerd[15689]: time="2021-12-16T12:00:26.343827266+02:00" level=error msg="Handler for POST /v1.41/plugins/elastic/elastic-logging-plugin:7.16.1/enable returned error: dial unix /run/docker/plugins/62d7ef0a88a0ad1e22f74da6241c3c7a636f9344cdbcb3b31bd3df97d36ab8e1/beatSocket.sock: connect: no such file or directory"
dec 16 12:00:26 glap2 dockerd[15689]: time="2021-12-16T12:00:26.566777455+02:00" level=error msg="Sending SIGTERM to plugin failed with error: process already finished: not found"
dec 16 12:00:26 glap2 dockerd[15689]: time="2021-12-16T12:00:26.572679156+02:00" level=info msg="ignoring event" container=62d7ef0a88a0ad1e22f74da6241c3c7a636f9344cdbcb3b31bd3df97d36ab8e1 module=libcontainerd namespace=plugins.moby topic=/tasks/delete type="*events.TaskDelete"
joepa37 commented 2 years ago

I have the exact error on v8.0.1

docker version `Client: Docker Engine - Community Version: 20.10.12 API version: 1.41 Go version: go1.16.12 Git commit: e91ed57 Built: Mon Dec 13 11:44:28 2021 OS/Arch: linux/arm64 Context: default Experimental: true

Server: Docker Engine - Community Engine: Version: 20.10.12 API version: 1.41 (minimum version 1.12) Go version: go1.16.12 Git commit: 459d0df Built: Mon Dec 13 11:43:05 2021 OS/Arch: linux/arm64 Experimental: false containerd: Version: 1.4.12 GitCommit: 7b11cfaabd73bb80907dd23182b9347b4245eb5d runc: Version: 1.0.2 GitCommit: v1.0.2-0-g52b36a2 docker-init: Version: 0.19.0 GitCommit: de40ad0`

Docker info `Client: Context: default Debug Mode: false Plugins: app: Docker App (Docker Inc., v0.9.1-beta3) buildx: Docker Buildx (Docker Inc., v0.7.1-docker)

Server: Containers: 4 Running: 4 Paused: 0 Stopped: 0 Images: 6 Server Version: 20.10.12 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: false userxattr: false Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc Default Runtime: runc Init Binary: docker-init containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d runc version: v1.0.2-0-g52b36a2 init version: de40ad0 Security Options: apparmor seccomp Profile: default Kernel Version: 5.13.0-1018-oracle Operating System: Ubuntu 20.04.3 LTS OSType: linux Architecture: aarch64 CPUs: 4 Total Memory: 23.43GiB Name: node.host.com Docker Root Dir: /var/lib/docker Debug Mode: false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false`

docker plugin install elastic/elastic-logging-plugin:8.0.1 `Plugin "elastic/elastic-logging-plugin:8.0.1" is requesting the following privileges:

journalctl -fu docker.service Mar 07 15:09:01 node.host.com dockerd[1032]: time="2022-03-07T15:09:01-06:00" level=error msg="standard_init_linux.go:228: exec user process caused: exec format error" plugin=e6d73dd98ce92e5b1d455eeedbe8c2ce00bd5d395462518044f30a986666744f Mar 07 15:09:01 node.host.com dockerd[1032]: time="2022-03-07T15:09:01.561533316-06:00" level=info msg="ignoring event" container=e6d73dd98ce92e5b1d455eeedbe8c2ce00bd5d395462518044f30a986666744f module=libcontainerd namespace=plugins.moby topic=/tasks/delete type="*events.TaskDelete" Mar 07 15:09:01 node.host.com dockerd[1032]: time="2022-03-07T15:09:01-06:00" level=error msg="standard_init_linux.go:228: exec user process caused: exec format error" plugin=e6d73dd98ce92e5b1d455eeedbe8c2ce00bd5d395462518044f30a986666744f Mar 07 15:09:14 node.host.comdockerd[1032]: time="2022-03-07T15:09:14.063254979-06:00" level=error msg="Sending SIGTERM to plugin failed with error: process already finished: not found" Mar 07 15:09:14 node.host.com dockerd[1032]: time="2022-03-07T15:09:14.063322699-06:00" level=error msg="Handler for POST /v1.41/plugins/elastic/elastic-logging-plugin:8.0.1/enable returned error: dial unix /run/docker/plugins/e6d73dd98ce92e5b1d455eeedbe8c2ce00bd5d395462518044f30a986666744f/beatSocket.sock: connect: no such file or directory" Mar 07 15:09:14 node.host.com dockerd[1032]: time="2022-03-07T15:09:14.154224253-06:00" level=error msg="Sending SIGTERM to plugin failed with error: process already finished: not found" Mar 07 15:09:14 node.host.com dockerd[1032]: time="2022-03-07T15:09:14.167991221-06:00" level=info msg="ignoring event" container=e6d73dd98ce92e5b1d455eeedbe8c2ce00bd5d395462518044f30a986666744f module=libcontainerd namespace=plugins.moby topic=/tasks/delete type="*events.TaskDelete"

I don't see any architecture restriction on https://www.docker.elastic.co/r/beats/elastic-logging-plugin Is it stable for aarch64?

ArturFormella commented 2 years ago

Exactly the same for me. My env:

➜  docker_knime git:(main) ✗ docker version
Client: Docker Engine - Community
 Version:           19.03.12
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        48a66213fe
 Built:             Mon Jun 22 15:45:49 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.12
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       48a66213fe
  Built:            Mon Jun 22 15:44:20 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
➜  docker_knime git:(main) ✗ uname -a
Linux DKXP 4.4.0-210-generic #242-Ubuntu SMP Fri Apr 16 09:57:56 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
wibbly13 commented 2 years ago

The issue seems to have been introduced in v7.13.0. I found it to be caused by a dependency to /lib64/ld-linux-x86-64.so.2 that's missing in the rootfs directory. This dependency can be installed by installing the libc6-compat package. This can be done by cloning the repo, adding RUN apk add libc6-compat here, and then building it (see instructions in beats docs).

I wonder why this issue doesn't have more people complaining?

WissameMekhilef commented 2 years ago

Glad to see that I'm not the only one having this issue.

botelastic[bot] commented 1 year ago

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

Firesphere commented 3 months ago

This error still happens on aarch64