search

linuxserver / docker-syslog-ng

GNU General Public License v3.0
51 stars 9 forks source link

[BUG] Can't create /var/log/messages and /var/log/messages-kv.log #19

Closed ialobanov closed 6 months ago

ialobanov commented 11 months ago

Is there an existing issue for this?

Current Behavior

After create a docker-compose file and run it, I get log:

2023-07-24 14:20:03.983945015  [2023-07-24T14:20:03.983913] Error opening file for writing; filename='/var/log/messages', error='No such file or directory (2)'
2023-07-24 14:20:03.984001063  [2023-07-24T14:20:03.983989] Error opening file for writing; filename='/var/log/messages-kv.log', error='No such file or directory (2)'
2023-07-24 14:21:04.040980883  [2023-07-24T14:21:04.040943] Error opening file for writing; filename='/var/log/messages', error='No such file or directory (2)'
2023-07-24 14:21:04.041039840  [2023-07-24T14:21:04.041022] Error opening file for writing; filename='/var/log/messages-kv.log', error='No such file or directory (2)'
2023-07-24 14:22:04.082063882  [2023-07-24T14:22:04.082028] Error opening file for writing; filename='/var/log/messages', error='No such file or directory (2)'
2023-07-24 14:22:04.082123633  [2023-07-24T14:22:04.082107] Error opening file for writing; filename='/var/log/messages-kv.log', error='No such file or directory (2)'

Expected Behavior

Work properly. I need to go inside a container and add rights 777 to files /var/log/messages and /var/log/messages-kv.log. After this it's work properly.

Steps To Reproduce

  1. Create docker-compose file.
  2. Run docker-compose file in detached mode.

Environment

- OS:Ubuntu 20.04.6 LTS (Focal Fossa)
- How docker service was installed: in terminal by official guied. (Docker version 24.0.4, build 3713ee1) (docker-compose version 1.25.0, build unknown)

CPU architecture

x86-64

Docker creation

version: '2.1'
services:
  syslog-ng:
    image: lscr.io/linuxserver/syslog-ng:latest
    container_name: syslog-ng
    environment:
      - PUID="1000"
      - PGID="1000"
      - TZ=Etc/UTC
    volumes:
      - ~/syslog-ng/config:/config
      - ~/syslog-ng/logs:/var/log
    ports:
      - 514:5514/udp
    restart: unless-stopped

Container logs

[migrations] started
[migrations] no migrations found
───────────────────────────────────────

      ██╗     ███████╗██╗ ██████╗
      ██║     ██╔════╝██║██╔═══██╗
      ██║     ███████╗██║██║   ██║
      ██║     ╚════██║██║██║   ██║
      ███████╗███████║██║╚██████╔╝
      ╚══════╝╚══════╝╚═╝ ╚═════╝

   Brought to you by linuxserver.io
───────────────────────────────────────

To support LSIO projects visit:
https://www.linuxserver.io/donate/

───────────────────────────────────────
GID/UID
───────────────────────────────────────

User UID:    1000
User GID:    1000
───────────────────────────────────────

[custom-init] No custom files found, skipping...
[ls.io-init] done.
[migrations] started
[migrations] no migrations found
usermod: no changes
───────────────────────────────────────

      ██╗     ███████╗██╗ ██████╗
      ██║     ██╔════╝██║██╔═══██╗
      ██║     ███████╗██║██║   ██║
      ██║     ╚════██║██║██║   ██║
      ███████╗███████║██║╚██████╔╝
      ╚══════╝╚══════╝╚═╝ ╚═════╝

   Brought to you by linuxserver.io
───────────────────────────────────────

To support LSIO projects visit:
https://www.linuxserver.io/donate/

───────────────────────────────────────
GID/UID
───────────────────────────────────────

User UID:    1000
User GID:    1000
───────────────────────────────────────

[custom-init] No custom files found, skipping...
[ls.io-init] done.
github-actions[bot] commented 11 months ago

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

thespad commented 11 months ago

You need to ensure the host permissions are correct for the folder you mount for your logs. The only path we manage permissions for is /config, as /var/log is just a default location but not mandatory, and if full of hundreds/thousands of logs then us running permissioning would noticeably slow down container startup.

ialobanov commented 11 months ago

Sorry, I am not sure that I understand you clearly. Can you explain to me what I need to check on host?

thespad commented 11 months ago

~/syslog-ng/logs needs to have the permissions for the user you're running the container as (1000:1000) to create files and folders otherwise you're going to get errors. Running a chmod inside the container will achieve mostly the same thing (it's the same filesystem) but a) it's not required if the permissions are correct before the container starts and b) 777 is not a safe permission set for files.

ialobanov commented 11 months ago

I use recommendations: 

environment:  
       - PUID=1000
       - PGID=1000

In my host UID and GID the same = 1000. After running docker-compse it's create folder in my home directory. Problem only with log's files inside container.

ialobanov commented 11 months ago

Tried again on the new server of Ubuntu 20.04 with new installation of docker and docker-compose. Problem the same.
# docker version:

Client: Docker Engine - Community
 Version:           24.0.5
 API version:       1.43
 Go version:        go1.20.6
 Git commit:        ced0996
 Built:             Fri Jul 21 20:35:23 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.5
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.6
  Git commit:       a61e2b4
  Built:            Fri Jul 21 20:35:23 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

# docker compose version:

Docker Compose version v2.20.2

Log docker-compose:

syslog-ng  | [migrations] started
syslog-ng  | [migrations] no migrations found
syslog-ng  | ───────────────────────────────────────
syslog-ng  |
syslog-ng  |
syslog-ng  |       ██╗     ███████╗██╗ ██████╗
syslog-ng  |       ██║     ██╔════╝██║██╔═══██╗
syslog-ng  |       ██║     ███████╗██║██║   ██║
syslog-ng  |       ██║     ╚════██║██║██║   ██║
syslog-ng  |       ███████╗███████║██║╚██████╔╝
syslog-ng  |       ╚══════╝╚══════╝╚═╝ ╚═════╝
syslog-ng  |
syslog-ng  |
syslog-ng  |    Brought to you by linuxserver.io
syslog-ng  | ───────────────────────────────────────
syslog-ng  |
syslog-ng  |
syslog-ng  | To support LSIO projects visit:
syslog-ng  | https://www.linuxserver.io/donate/
syslog-ng  |
syslog-ng  |
syslog-ng  | ───────────────────────────────────────
syslog-ng  | GID/UID
syslog-ng  | ───────────────────────────────────────
syslog-ng  |
syslog-ng  |
syslog-ng  | User UID:    1000
syslog-ng  | User GID:    1000
syslog-ng  | ───────────────────────────────────────
syslog-ng  |
syslog-ng  |
syslog-ng  | [custom-init] No custom files found, skipping...
syslog-ng  | [ls.io-init] done.

Log /config/log/current:

2023-07-25 06:18:21.071389006  [2023-07-25T06:18:21.071357] Error opening file for writing; filename='/var/log/messages', error='No such file or directory (2)'
2023-07-25 06:18:21.071465991  [2023-07-25T06:18:21.071451] Error opening file for writing; filename='/var/log/messages-kv.log', error='No such file or directory (2)'
2023-07-25 06:19:21.072456252  [2023-07-25T06:19:21.072415] Error opening file for writing; filename='/var/log/messages', error='No such file or directory (2)'
2023-07-25 06:19:21.072459553  [2023-07-25T06:19:21.072437] Error opening file for writing; filename='/var/log/messages-kv.log', error='No such file or directory (2)'
2023-07-25 06:20:21.132403898  [2023-07-25T06:20:21.132359] Error opening file for writing; filename='/var/log/messages', error='No such file or directory (2)'
2023-07-25 06:20:21.132406772  [2023-07-25T06:20:21.132384] Error opening file for writing; filename='/var/log/messages-kv.log', error='No such file or directory (2)'
2023-07-25 06:21:21.189828405  [2023-07-25T06:21:21.189747] Error opening file for writing; filename='/var/log/messages', error='No such file or directory (2)'
2023-07-25 06:21:21.189831228  [2023-07-25T06:21:21.189792] Error opening file for writing; filename='/var/log/messages-kv.log', error='No such file or directory (2)'
2023-07-25 06:22:21.223707234  [2023-07-25T06:22:21.223664] Error opening file for writing; filename='/var/log/messages', error='No such file or directory (2)'
2023-07-25 06:22:21.223710156  [2023-07-25T06:22:21.223687] Error opening file for writing; filename='/var/log/messages-kv.log', error='No such file or directory (2)'
2023-07-25 06:23:21.275037882  [2023-07-25T06:23:21.274994] Error opening file for writing; filename='/var/log/messages', error='No such file or directory (2)'
2023-07-25 06:23:21.275041121  [2023-07-25T06:23:21.275017] Error opening file for writing; filename='/var/log/messages-kv.log', error='No such file or directory (2)'
2023-07-25 06:24:21.308500053  [2023-07-25T06:24:21.308456] Error opening file for writing; filename='/var/log/messages', error='No such file or directory (2)'
2023-07-25 06:24:21.308502837  [2023-07-25T06:24:21.308479] Error opening file for writing; filename='/var/log/messages-kv.log', error='No such file or directory (2)'
LinuxServer-CI commented 10 months ago

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.

jeremywillans commented 10 months ago

@thespad, I am also seeing this issue for setting up a new syslog-ng container.

LinuxServer-CI commented 9 months ago

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.

drizuid commented 9 months ago

OP: What is the output of ls -asnl ~/syslog-ng/logs What is the output of id 1000

LinuxServer-CI commented 8 months ago

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.

samweisgamdschie commented 5 months ago

Hi!

I am facing the exact same issue on debian 12 host with rootless podman.

I am too let it run as the first configured user (id 1000). My output according to ls -asnl ~/syslog-ng/logs is:

$ ls -asnl syslog-ng-*
syslog-ng-config:
insgesamt 24
 0 drwxr-xr-x 1 100999 100999   120 31. Jän 22:28 .
 0 drwxr-xr-x 1   1000   1000   194 25. Jän 00:06 ..
 0 drwx------ 1 100999 100999    86 31. Jän 21:44 log
 4 -rw-r--r-- 1 100999 100999   685 18. Jän 19:58 syslog-ng.conf
 0 srwxr-xr-x 1 100999 100999     0 31. Jän 22:28 syslog-ng.ctl
16 -rw------- 1 100999 100999 16384 31. Jän 22:28 syslog-ng.persist
 4 -rw-r--r-- 1 100999 100999     4 31. Jän 22:28 syslog-ng.pid

syslog-ng-log:
insgesamt 0
0 drwxr-xr-x 1 1000 1000   4 31. Jän 22:32 .
0 drwxr-xr-x 1 1000 1000 194 25. Jän 00:06 ..
0 -rw-r--r-- 1 1000 1000   0 31. Jän 22:32 eh

and id 1000 gives:

$ id 1000
uid=1000(myuser) gid=1000(myuser) Gruppen=1000(myuser),24(cdrom),25(floppy),29(audio),30(dip),44(video),46(plugdev),100(users),106(netdev)

Finally I started a bash inside the running container:

$ podman exec -it syslog-ng bash
root@da83f31c1846:/# 
root@da83f31c1846:/# 
root@da83f31c1846:/# 
root@da83f31c1846:/# 
root@da83f31c1846:/# 
root@da83f31c1846:/# cd /var/log/
root@da83f31c1846:/var/log# ls
root@da83f31c1846:/var/log# touch eh
root@da83f31c1846:/var/log# ls
eh
root@da83f31c1846:/var/log# ls -la
total 0
drwxr-xr-x 1 root root  4 Jan 31 22:32 .
drwxr-xr-x 1 root root 86 Jan 13 14:30 ..
-rw-r--r-- 1 root root  0 Jan 31 22:32 eh
root@da83f31c1846:/var/log#

As you can see, touching a file in /var/log DOES work. And the permissions on the host system are correct:

$ ls syslog-ng-log/eh  -l
-rw-r--r-- 1 myuser myuser 0 31. Jän 22:32 syslog-ng-log/eh

To deliver more information:

Startup logs:

$ tail -f podman-volumes/syslog-ng-config/logcurrent -n0
2024-01-31 22:47:45.724750344  [2024-01-31T22:47:45.724707] Accepting connections; addr='AF_INET(0.0.0.0:6601)'
2024-01-31 22:47:45.725150397  [2024-01-31T22:47:45.725124] syslog-ng starting up; version='4.1.1'
2024-01-31 22:47:55.912031290  [2024-01-31T22:47:55.911983] Error opening file for writing; filename='/var/log/messages', error='No such file or directory (2)'
2024-01-31 22:47:55.912075134  [2024-01-31T22:47:55.912053] Error opening file for writing; filename='/var/log/messages-kv.log', error='No such file or directory (2)'

And of course the service part of the podman-compose.yaml:

services:
  syslog-ng:
    # docs: https://hub.docker.com/r/linuxserver/syslog-ng
    cap_add:
      - NET_ADMIN
    container_name: syslog-ng
    image: docker.io/linuxserver/syslog-ng
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/Vienna
    volumes:
      - /home/myuser/srv/podman-volumes/syslog-ng-config:/config
      - /home/myuser/srv/podman-volumes/syslog-ng-log:/var/log
    ports:
      - 514:5514/udp
      # - 601:6601/tcp # Syslog TCP
      # - 6514:6514/tcp # Syslog TLS
    restart: unless-stopped

EDIT: added version information:

$ cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

$ podman --version
podman version 4.3.1

$ podman-compose --version
podman-compose version: 1.0.6
['podman', '--version', '']
using podman version: 4.3.1
podman-compose version 1.0.6
podman --version 
podman version 4.3.1
exit code: 0

Any help is appreciated :)

samweisgamdschie commented 5 months ago

Should I open another issue rather than commenting here?

drizuid commented 5 months ago

it would make sense to open a new issue rather than posting in a closed issue. However, we do not test or support podman nor do we support rootless on any platform, though it can be made to work.