Closed chrisns closed 2 years ago
Thank you for opening the issue, I'm happy to look into it but I'm lacking Apple hardware to give it a test. I assume it's not possible to export the docker machine VM to a kvm or virtualbox compatible format, isn't it?
About the log enricher issue:
About the ebpf recorder:
/etc/os-release
or /usr/lib/os-release
available within the VM?OS not found in btf map:
, maybe I can improve thatexporting, not sure, podman machine has the same issue, is a different vm, but is fedora core 35 which is probably a bit easier to reproduce and inspect. have some free aws credit i'm happy to throw at a mac there to help debug if it helps?
auditd -f
Config file /etc/audit/auditd.conf opened for parsing
local_events_parser called with: yes
write_logs_parser called with: yes
log_file_parser called with: /var/log/audit/audit.log
log_group_parser called with: root
log_format_parser called with: ENRICHED
flush_parser called with: INCREMENTAL_ASYNC
freq_parser called with: 50
max_log_size_parser called with: 8
num_logs_parser called with: 5
priority_boost_parser called with: 4
name_format_parser called with: NONE
max_log_size_action_parser called with: ROTATE
space_left_parser called with: 75
space_action_parser called with: SYSLOG
verify_email_parser called with: yes
action_mail_acct_parser called with: root
admin_space_left_parser called with: 50
admin_space_left_action_parser called with: SUSPEND
disk_full_action_parser called with: SUSPEND
disk_error_action_parser called with: SUSPEND
use_libwrap_parser called with: yes
tcp_listen_queue_parser called with: 5
tcp_max_per_addr_parser called with: 1
tcp_client_max_idle_parser called with: 0
transport_parser called with: TCP
krb5_principal_parser called with: auditd
GSSAPI support is not enabled, ignoring value at line 33
distribute_network_parser called with: no
q_depth_parser called with: 1200
overflow_action_parser called with: SYSLOG
max_restarts_parser called with: 10
plugin_dir_parser called with: /etc/audit/plugins.d
eoe_timeout_parser called with: 2
No plugins found, not dispatching events
type=DAEMON_START msg=audit(1641892807.972:3574): op=start ver=3.0.6 format=enriched kernel=5.10.76-linuxkit auid=4294967295 pid=9677 uid=0 ses=4294967295 res=successAUID="unset" UID="root"
config_manager init complete
Error sending status request (Operation not permitted)
Error sending enable request (Operation not permitted)
type=DAEMON_ABORT msg=audit(1641892807.972:3575): op=set-enable auid=4294967295 pid=9677 uid=0 ses=4294967295 res=failedAUID="unset" UID="root"
Unable to set initial audit startup state to 'enable', exiting
The audit daemon is exiting.
Error setting audit daemon pid (Operation not permitted)
syslog looks about the same to my naive eyes:
$ uname -ra
Linux docker-desktop 5.10.76-linuxkit #1 SMP PREEMPT Mon Nov 8 11:22:26 UTC 2021 aarch64 Linux
$ cat /etc/os-release
PRETTY_NAME="Docker Desktop"`
$ cat /usr/lib/os-release
cat: can't open '/usr/lib/os-release': No such file or directory
$ uname -ra
Linux localhost.localdomain 5.15.12-200.fc35.aarch64 #1 SMP Wed Dec 29 14:47:47 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
$ cat /etc/os-release
NAME="Fedora Linux"
VERSION="35.20220103.2.0 (CoreOS)"
ID=fedora
VERSION_ID=35
VERSION_CODENAME=""
PLATFORM_ID="platform:f35"
PRETTY_NAME="Fedora CoreOS 35.20220103.2.0"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:35"
HOME_URL="https://getfedora.org/coreos/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora-coreos/"
SUPPORT_URL="https://github.com/coreos/fedora-coreos-tracker/"
BUG_REPORT_URL="https://github.com/coreos/fedora-coreos-tracker/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=35
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=35
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="CoreOS"
VARIANT_ID=coreos
OSTREE_VERSION='35.20220103.2.0'
DEFAULT_HOSTNAME=localhost
@chrisns can we close this one or do you wanna keep it open until the next release?
How long till the release where this is 'real'?
This week is my target goal. :)
cool! either close or keep open if it helps motivate towards that goal π cheers π
can close this now right?
π I don't appear to be able to get the profile recorder to work with docker-desktop or podman machine in combination with kind|k3d|minikube (minikube using --driver podman|docker)
I think the issue is the lack of syslog/auditd running in the host vm, and then maybe something to do with correlating between pid namespaces.
I've made it work using docker desktop's built in k8s service, which is different to kind|k3d|minikube by running the k8s api-server and other components and workload at the top level without nesting - at least I think thats whats going on by doing
docker ps
I can see everything. I don't think these issues are specific to being on a mac/m1/arm though I've definitely made it harder for myself this way.I documented my work so far here: https://github.com/appvia/security-profiles-operator-demo
the eBPF recorder leaves me with these logs:
I think it'll be reasonably easy to reproduce by running syslog in a container rather than trying to look at the host's
/var/log/syslog
or/var/log/audit/audit.log
so perhaps as a feature request rather than the log-enricher looking for the logs in hard coded locations, if it tailed the kernel buffer ring itself and handled mapping between the pid namespaces through some magic that at least bring some consistency.In a thread on the Kubernetes slack I raised this https://kubernetes.slack.com/archives/C013FQNB0A2/p1641572536002500 @pjbgf suggested it might be possible to make the SPO better aware of pids by mounting a the host
/proc
https://github.com/kubernetes-sigs/security-profiles-operator/blob/d49ab905b22a2afc25a6c30bffe547c4fff84276/internal/pkg/util/containers.go#L55 and using that.I tried that https://github.com/chrisns/security-profiles-operator/commit/dc1f948caa8fe16c33016aac6297e02a0bfb3e1b but no different results π’ but then I really have no idea what I'm doing π€£
Thoughts welcome, or better yet, tell me I'm being an an idiot and done it all wrong π€£