fluent / fluentd-kubernetes-daemonset

Fluentd daemonset for Kubernetes and it Docker image
Apache License 2.0
1.25k stars 981 forks source link

Tries to place lock file on read-only filesystem #1453

Closed MartinEmrich closed 8 months ago

MartinEmrich commented 11 months ago

After moving to 1.16 from 1.14, fluentd-graylog no longer starts.

I receive this message:

2023-08-03 14:47:31 +0000 [info]: spawn command to main:  cmdline=["/usr/local/bin/ruby", "-Eascii-8bit:ascii-8bit", "/fluentd/vendor/bundle/ruby/3.1.0/bin/fluentd", "-c", "/fluentd/etc/fluent.conf", "-p", "/fluentd/plugins", "--gemfile", "/fluentd/Gemfile", "--under-supervisor"]
system temporary path is world-writable: /tmp
/tmp is world-writable: /tmp
Unexpected error Read-only file system @ dir_s_mkdir - /home/fluent/fluentd-lock-20230803-7-1vor
  /usr/local/lib/ruby/3.1.0/tmpdir.rb:92:in `mkdir'
  /usr/local/lib/ruby/3.1.0/tmpdir.rb:92:in `block in mktmpdir'
  /usr/local/lib/ruby/3.1.0/tmpdir.rb:144:in `create'
  /usr/local/lib/ruby/3.1.0/tmpdir.rb:90:in `mktmpdir'
  /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:47:in `before_run'
  /fluentd/vendor/bundle/ruby/3.1.0/gems/serverengine-2.3.2/lib/serverengine/server.rb:125:in `main'
  /fluentd/vendor/bundle/ruby/3.1.0/gems/serverengine-2.3.2/lib/serverengine/daemon.rb:119:in `main'
  /fluentd/vendor/bundle/ruby/3.1.0/gems/serverengine-2.3.2/lib/serverengine/daemon.rb:68:in `run'
  /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:796:in `supervise'
  /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:582:in `run_supervisor'
  /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.16.2/lib/fluent/command/fluentd.rb:352:in `<top (required)>'
  <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.16.2/bin/fluentd:15:in `<top (required)>'
  /fluentd/vendor/bundle/ruby/3.1.0/bin/fluentd:25:in `load'
  /fluentd/vendor/bundle/ruby/3.1.0/bin/fluentd:25:in `<main>'

I run it on a read-only filesystem, but provide a writeable /tmp.

I would suggest that in a kubernetes pod, a lock file is unecessary.

MartinEmrich commented 11 months ago

Providing another emptyDir in /home/fluent brings me back to https://github.com/fluent/fluentd-kubernetes-daemonset/issues/1393 :(

MartinEmrich commented 11 months ago

A workaround: use an initcontainer to chmod the directories, e.g:

...
      initContainers:
        - name: chmod-tempdirs
          image: fluent/fluentd-kubernetes-daemonset:v1.16-debian-graylog-1
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            # runAsNonRoot: true
            capabilities:
              drop:
                - all
          command: "/bin/sh"
          args:
          - "-c"
          - "chmod o-rwx /home/fluent /tmp"
          volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: homefluent
              mountPath: /home/fluent
...
github-actions[bot] commented 8 months ago

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

MartinEmrich commented 8 months ago

I won't bother with the (IMHO annoying) I-dont-care-github-bot... I switched to https://vector.dev/ .