search

GoogleContainerTools / kaniko

Build Container Images In Kubernetes
Apache License 2.0
14.77k stars 1.44k forks source link

Memory Leak in warmer #2754

Closed mxbossard closed 10 months ago

mxbossard commented 1 year ago

Actual behavior When we warm an image it's fully loaded in RAM before it's written on disk. The TarWriter is a bytes.Buffer here: https://github.com/GoogleContainerTools/kaniko/blob/8c09efff4c45d4a7edeb4960b03eb28da3c0a213/pkg/cache/warm.go#L68 Then the files are written here: https://github.com/GoogleContainerTools/kaniko/blob/8c09efff4c45d4a7edeb4960b03eb28da3c0a213/pkg/cache/warm.go#L84

Expected behavior The downloaded file should be streamed into a file on disk with a FileWritter.

To Reproduce warmer -i

Triage Notes for the Maintainers

Description Yes/No
Please check if this a new feature you are proposing
  • - [No]
aaron-prindle commented 1 year ago

Thank you for flagging this @mxbossard! Would you want to drive a PR to fix this issue - changing the warmer to stream files vs loading them all into RAM first?

mxbossard commented 1 year ago

I gave it a try and proposed a PR. I added a shell script in scripts dir to test the warmer in a docker container within boxed memory conditions. I could reproduce the oom error before the fix and validate the fix.