search

elastic / elastic-agent

Elastic Agent - single, unified way to add monitoring for logs, metrics, and other types of data to a host.
Other
22 stars 143 forks source link

Fix permissions for Ironbank images #4539

Open michalpristas opened 7 months ago

michalpristas commented 7 months ago

group permissions are missing.

RUN chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/metricbeat && \
chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/fleet-server && \
chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/auditbeat && \
chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/cloudbeat && \
chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/endpoint-security && \
chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/heartbeat && \
chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/packetbeat && \
chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/osquerybeat && \
chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/filebeat
USER root
RUN chown -R root /usr/share/elastic-agent/data/elastic-agent-*/components/mod
elasticmachine commented 7 months ago

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

zedtran commented 6 months ago

^ are listed above, but should the ones below that aren't indicated also be included? You can find them with:

find /usr/share/elastic-agent/data/elastic-agent-*/components -maxdepth 1 -type f -not -name "*.*"

Would the below work?

find /usr/share/elastic-agent/data/elastic-agent-*/components -maxdepth 4 -type d -exec chmod 0660 {} \;

and 

find /usr/share/elastic-agent/data/elastic-agent-*/components -maxdepth 1 -type f -not -name "*.*" -exec chmod 0755 {} \;
zedtran commented 6 months ago

chown -R root /usr/share/elastic-agent/data/elastic-agent-*/components/mod

chown: cannot access '/usr/share/elastic-agent/data/elastic-agent-*/components/mod': No such file or directory

Eric-Domeier commented 6 months ago

After further testing with the initial dockerfile I also ran into the following error(s) when launching elastic agent (ironbank)

{"log.level":"error","@timestamp":"2024-05-06T22:34:31.508Z","message":"Failed to list light metricsets for module kibana: getting metricsets for module 'kibana': loading light module 'kibana' definition: loading module configuration from '/usr/share/elastic-agent/data/elastic-agent-de80b0/components/module/kibana/module.yml': config file (\"/usr/share/elastic-agent/data/elastic-agent-de80b0/components/module/kibana/module.yml\") can only be writable by the owner but the permissions are \"-rw-rw----\" (to fix the permissions use: 'chmod go-w /usr/share/elastic-agent/data/elastic-agent-de80b0/components/module/kibana/module.yml')","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"http/metrics-monitoring","type":"http/metrics"},"log":{"source":"http/metrics-monitoring"},"ecs.version":"1.6.0","log.logger":"registry.lightmodules","log.origin":{"file.line":145,"file.name":"mb/lightmodules.go","function":"github.com/elastic/beats/v7/metricbeat/mb.(*LightModulesSource).ModulesInfo"},"service.name":"metricbeat","ecs.version":"1.6.0"}

There were various .yml files that seemed to have the group writeable permissions that it didn't like.

I fixed this with the following Dockerfile

FROM registry1.dso.mil/ironbank/elastic/beats/elastic-agent:8.12.2
RUN chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/metricbeat && \
    chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/fleet-server && \
    chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/auditbeat && \
    chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/cloudbeat && \
    chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/endpoint-security && \
    chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/heartbeat && \
    chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/packetbeat && \
    chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/osquerybeat && \
    chmod 755 /opt/elastic-agent/data/elastic-agent-*/components/filebeat && \
    find /usr/share/elastic-agent/data/elastic-agent-*/components/module/ -name "*.yml" -exec chmod go-w {} \;

USER root
RUN chown -R root /usr/share/elastic-agent/data/elastic-agent-*/components/module/
USER elastic-agent
ENTRYPOINT ["/tinit", "--", "/usr/local/bin/docker-entrypoint"]
CMD [""]

I will try to test this with @zedtran 's suggestions

FROM registry1.dso.mil/ironbank/elastic/beats/elastic-agent:8.12.2
RUN find /usr/share/elastic-agent/data/elastic-agent-*/components -maxdepth 4 -type d -exec chmod 0660 {} \; && \
    find /usr/share/elastic-agent/data/elastic-agent-*/components -maxdepth 1 -type f -not -name "*.*" -exec chmod 0755 {} \; && \
    find /usr/share/elastic-agent/data/elastic-agent-*/components/module/ -name "*.yml" -exec chmod go-w {} \;

USER root
RUN chown -R root /usr/share/elastic-agent/data/elastic-agent-*/components/module/
USER elastic-agent
ENTRYPOINT ["/tinit", "--", "/usr/local/bin/docker-entrypoint"]
CMD [""]
strawgate commented 6 months ago

If we need the container images to run as read-only anyway, is it reasonable to actually remove root write permissions here? Or is that a change for later?

Eric-Domeier commented 6 months ago

@zedtran

My initial testing results with the container unable to start, the find command

find /usr/share/elastic-agent/data/elastic-agent-*/components -maxdepth 1 -type f -not -name "*.*" -exec chmod 0755 {} \;

Changes the following files

/usr/share/elastic-agent/data/elastic-agent-de80b0/components/apm-server /usr/share/elastic-agent/data/elastic-agent-de80b0/components/auditbeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/cloud-defend /usr/share/elastic-agent/data/elastic-agent-de80b0/components/cloudbeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/endpoint-security /usr/share/elastic-agent/data/elastic-agent-de80b0/components/filebeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/fleet-server /usr/share/elastic-agent/data/elastic-agent-de80b0/components/heartbeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/metricbeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/osquerybeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/osqueryd /usr/share/elastic-agent/data/elastic-agent-de80b0/components/packetbeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/pf-elastic-collector /usr/share/elastic-agent/data/elastic-agent-de80b0/components/pf-elastic-symbolizer /usr/share/elastic-agent/data/elastic-agent-de80b0/components/pf-host-agent

Container fails to start with the following error

Error: failed to detect inputs and outputs: failed reading spec /usr/share/elastic-agent/data/elastic-agent-de80b0/components/apm-server.spec.yml: open /usr/share/elastic-agent/data/elastic-agent-de80b0/components/apm-server.spec.yml: permission denied

cmacknz commented 6 months ago

There is no reason the module YAML files need to be writable in our container, or really at all when built as part of agent. This is a permanent change we could make to the agent container.

For 8.14+ we could do this as part of the agentbeat build which added control of these fields in https://github.com/elastic/beats/pull/39278. For 7.17.x we have to do this in a different way.

zedtran commented 6 months ago

@zedtran

My initial testing results with the container unable to start, the find command

find /usr/share/elastic-agent/data/elastic-agent-*/components -maxdepth 1 -type f -not -name "*.*" -exec chmod 0755 {} \;

Changes the following files

/usr/share/elastic-agent/data/elastic-agent-de80b0/components/apm-server /usr/share/elastic-agent/data/elastic-agent-de80b0/components/auditbeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/cloud-defend /usr/share/elastic-agent/data/elastic-agent-de80b0/components/cloudbeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/endpoint-security /usr/share/elastic-agent/data/elastic-agent-de80b0/components/filebeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/fleet-server /usr/share/elastic-agent/data/elastic-agent-de80b0/components/heartbeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/metricbeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/osquerybeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/osqueryd /usr/share/elastic-agent/data/elastic-agent-de80b0/components/packetbeat /usr/share/elastic-agent/data/elastic-agent-de80b0/components/pf-elastic-collector /usr/share/elastic-agent/data/elastic-agent-de80b0/components/pf-elastic-symbolizer /usr/share/elastic-agent/data/elastic-agent-de80b0/components/pf-host-agent

Container fails to start with the following error

Error: failed to detect inputs and outputs: failed reading spec /usr/share/elastic-agent/data/elastic-agent-de80b0/components/apm-server.spec.yml: open /usr/share/elastic-agent/data/elastic-agent-de80b0/components/apm-server.spec.yml: permission denied

@Eric-Domeier, the find command you referenced gives user/group permissions to execute on the GO binaries you indicated above. Neither of the find commands I suggested appear to account for your situation as it pertains to file apm-server.spec.yml. It appears you need something like (or similar to):

find /usr/share/elastic-agent/data/elastic-agent-*/components -maxdepth 4 -type f -name "*.yml" -exec chmod 0664 {} \;

I did not encounter the errors you observed related to the noted .yml files, so I did not include such a recommendation (above). I recommend possibly also checking environment variable values for $BEAT_SETUID_AS (whoami) and that $ELASTIC_UID and/or $ELASTIC_GID are expected values based on the started UID/GID in container. Since I don't have any use (currently) for Elastic Synthetics monitoring, I start with the default OCI spec UID & GID combination (1000 and 0, respectively).

I hope that helps.

zedtran commented 6 months ago

Edit: @Eric-Domeier, as there are other file extensions like .zip, .csv, .json, .ext, .disabled, .md, and .jar, it may benefit you to run one of the following instead (It's pretty much the same as the find command I used for type d (directories) and excludes those component GO binaries):

find /usr/share/elastic-agent/data/elastic-agent-*/components -maxdepth 4 -type f -name "*.*" -exec chmod 0640 {} \;

Below would override the chmod go-w at the tail end of your RUN directive in the Dockerfile, so I would use that only if your UID/GID combo doesn't present you with the same issue(s).

find /usr/share/elastic-agent/data/elastic-agent-*/components -maxdepth 4 -type f -name "*.*" -exec chmod 0660 {} \;
elasticmachine commented 5 months ago

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)