seccomp profiles failing to install with "Too long" error

gsstuart commented 1 month ago

What happened:

After concluding some seccomp ProfileRecordings, two of the profiles failed to install; one has an empty status and the other is stuck in Terminating:

$ k get sp -A | grep -v Installed
NAMESPACE                          NAME                                           STATUS        AGE
production-campaigns               production-campaigns-component                               5d2h
production-dialer-api              production-dialer-api-setup-emptydir           Terminating   5d2h

The following errors are seen in the spod logs, indicating that the annotations could not be updated for these profiles:

E1002 20:04:51.351691    7746 seccompprofile.go:448] "merge base profile" err="update seccomp profile annotations: SeccompProfile.security-profiles-operator.x-k8s.io \"production-campaigns-component\" is invalid: metadata.annotations: Too long: must have at most 262144 bytes" logger="seccomp-spod" profile="production-campaigns-component" namespace="production-campaigns"
E1002 20:05:01.859993    7746 seccompprofile.go:448] "merge base profile" err="update seccomp profile annotations: SeccompProfile.security-profiles-operator.x-k8s.io \"production-dialer-api-setup-emptydir\" is invalid: metadata.annotations: Too long: must have at most 262144 bytes" logger="seccomp-spod" profile="production-dialer-api-setup-emptydir" namespace="production-dialer-api"

What you expected to happen:

All profiles to be in Installed state.

How to reproduce it (as minimally and precisely as possible):

Run profile recordings long enough (~3 days in this case) to generate seccompprofile objects whose JSON representations are longer than 262144 bytes. Although interestingly, we also have some other SPs longer than that which did install without issue and did not generate this error.

Anything else we need to know?:

Environment:

Cloud provider or hardware configuration: AWS EKS
OS (e.g: cat /etc/os-release): Amazon Linux 2
Kernel (e.g. uname -a): 5.10.217-205.860.amzn2.x86_64
Others:

ccojocar commented 1 month ago

Is is possible to post an example? What makes the profile so large? Are there any duplicated syscalls which you see in the profile? Thanks

gsstuart commented 1 month ago

Hi @ccojocar, sure, I'll upload an example here... there are indeed a lot of duplicated syscalls, although all of the generated profiles have been like that. I assumed that was just a side effect of the merging : ) Here's the ProfileRecording object that I've used:

---
apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
kind: ProfileRecording
metadata:
  name: NAME
  namespace: NAMESPACE
spec:
  kind: SeccompProfile
  disableProfileAfterRecording: false
  recorder: logs
  mergeStrategy: containers
  podSelector:
    matchLabels:
      app.kubernetes.io/managed-by: LABEL_VALUE

And here's one of the larger resulting seccomp profiles: sp-example.yml.txt

kubernetes-sigs / security-profiles-operator