logDNA supertenant agent volume mounts

[somnathpathak]

To accommodate multiple individual services in a single OpenShift cluster, I need to deploy multiple logDNA supertenant agents for each service in its own namespace.

These individual agents would then scrape logs from sub-directories under /var/log for example: agent-1 scrapes logs from /var/log/my-service-1, similarly agent-2 scrapes logs from /var/log/my-service-2.

However, while configuring these agents I also need to maintain isolation between each of them due to compliance requirements. This means I do not want these agents to unnecessarily share volumes, which I came across here while defining the agent yaml.

Please advise on why we need the below mentioned volumes and which I could drop from my configuration.

        - name: varlog
          hostPath:
            path: /var/log
        - name: vardata
          hostPath:
            path: /var/data
        - name: varliblogdna
          hostPath:
            path: /var/lib/logdna
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: mnt
          hostPath:
            path: /mnt
        - name: osrelease
          hostPath:
            path: /etc/os-release
        - name: logdnahostname
          hostPath:
            path: /etc/hostname

I understand for configuring lookback in the agent I need the /var/lib/logdna volume but not particularly sure on the other volumes. I specifically feel like I could drop the /var/data and /var/lib/docker/containers as these shared directories between the deployed cluster services and need not be mounted for compliance sake.

[somnathpathak]

@dkhokhlov Could you please guide me here.

[somnathpathak]

UPDATE:

The agent configuration works fine with just the two below-mentioned mounts as well. Should I still consider using other volumes as well?

      volumes:
        - name: varlog
          hostPath:
            path: /var/log/myservice
            type: ''
        - name: varliblogdna
          hostPath:
            path: /var/lib/logdna
            type: ''

[somnathpathak]

CC @c-nixon ^

[somnathpathak]

Can someone from the team please look into it and provide the requested guidance?

[dkhokhlov]

UPDATE:

The agent configuration works fine with just the two below-mentioned mounts as well. Should I still consider using other volumes as well?

      volumes:
        - name: varlog
          hostPath:
            path: /var/log/myservice
            type: ''
        - name: varliblogdna
          hostPath:
            path: /var/lib/logdna
            type: ''

keep varliblogdna different and the rest volumes the same. the varliblogdna is used to store DB with information about processed offsets of log files (inodes), not shareable.

[somnathpathak]

@dkhokhlov

        - name: varlog
          hostPath:
            path: /var/log
        - name: vardata
          hostPath:
            path: /var/data
        - name: varliblogdna
          hostPath:
            path: /var/lib/logdna
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: mnt
          hostPath:
            path: /mnt
        - name: osrelease
          hostPath:
            path: /etc/os-release
        - name: logdnahostname
          hostPath:
            path: /etc/hostname

Do you want me to keep all the volumes from the above list, and only differentiate for:

1. /var/lib/logdna (LOGDNA_DB_PATH) with something like /var/lib/logdna/myservice, and
2. /var/log/myservice (individual service logs directory in the shared cluster)

[dkhokhlov]

correct. note: you may need to adjust exclusion rules to eliminate duplicates from default logdir /var/log (cannot override it in env var, only in file config). also disable journald in one of two.

[somnathpathak]

@dkhokhlov Thank you. My configuration would now look something like below:

        - name: varlogmyservice
          hostPath:
            path: /var/log/myservice
        - name: vardata
          hostPath:
            path: /var/data
        - name: varliblogdna
          hostPath:
            path: /var/lib/logdna/myservice
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: mnt
          hostPath:
            path: /mnt
        - name: osrelease
          hostPath:
            path: /etc/os-release
        - name: logdnahostname
          hostPath:
            path: /etc/hostname

I would allow the services to mount only their respective sub-directories under /var/log parent directory e.g. /var/log/myservice and write logs into it. This means no service onboarded onto the shared cluster would be writing logs directly into the parent directory /var/log, thus eliminating the chances of duplicate log entries.

Also, with respect to the Journald logs, currently I do not wish to capture it. So the configuration of the same should be irrelevant in my use-case.

Please correct/confirm if my above shared understanding is correct.

[dkhokhlov]

lgtm