aquasecurity / trivy-operator

Kubernetes-native security toolkit
https://aquasecurity.github.io/trivy-operator/latest
Apache License 2.0
1.29k stars 214 forks source link

needlessly using follow option when retrieving logs of completed job #2264

Open jrhunger opened 2 months ago

jrhunger commented 2 months ago

What steps did you take and what happened: Seeing error "illegal base64 data at input byte 5761" in trivy-operator logs.
Looking at the completed job in question, we are able to successfully query and decode the logs:

$ kubectl -n trivy-system logs scan-vulnerabilityreport-d4856fb89-jbzzz  > x
$ wc -c x
5755 x
## note that the illegal input byte 5761 is > the size of this file 5755
$ base64 --decode < x > y
$ head -3 y
{
  "SchemaVersion": 2,
  "CreatedAt": "2024-09-17T09:43:51.613221809Z",

However if we add the -f option to kubectl logs, we get an extra line at the end:

MOo50krocvh7q08U2UkR+LzJgnsPQtfXuwggOKzv/V0EkLKRJH4Peyf9VD/NFLEkT/4x/pWf23/o
82xjGG/1/P9/+L7Ob/x3H/r/8VVPF4/7f7OLpw6dBS/1pUrrVJx/T/zuahnSGIzw5Q/wcrT7KmMw
/4u5IpwoSFKbvOGA
failed to create fsnotify watcher: too many open files

OK, granted this is an issue with our system, however, it also implies that trivy-operator is using the "follow" option when pulling the logs from a completed job, which shouldn't be necessary (no new log entries will be created for a completed job).

What did you expect to happen: I expect that trivy-operator can retrieve the logs from a completed job even if there is an issue on the system that causes an extra non-base64 line to be added when retrieving logs in Follow mode.

Anything else you would like to add: Looking in the logs, i see Follow: true in GetLogsByJobAndContainerName That is only called in pkg/configauditreport/controller/nodecollector.go and pkg/vulnerabilityreport/controller/scanjob.go, in each case inside a function called "processCompleteScanJob" (although there is an intermediate function on the 2nd one)

So if this is always called on Completed jobs, it seems the Follow: true is not necessary.

Environment:

github-actions[bot] commented 1 week ago

This issue is stale because it has been labeled with inactivity.

jrhunger commented 1 week ago

Should stay active. Pull request pending review.