search

nuvo / skbn

Copy files and directories between Kubernetes and cloud storage
Apache License 2.0
74 stars 76 forks source link

OMMkiller terminated backup process #16

Open emelianrus opened 4 years ago

emelianrus commented 4 years ago

I'm trying use cron job to copy files from kubernetes to S3 and in kubernetes and got pod error OMMKilled Looks like it slowly increase virtual memory and reaching pod limits.

Container limits set to: cpu 3. Memory: 5Gb amount of copy files: 3401311 changing buffer memory didn't help Am i doing something wrong? or its memory leak issue?

emelianrus commented 4 years ago

pprof.pdf

diasjorge commented 4 years ago

I've had to play with the parallel argument. Setting parallel to 0 has caused me problems. In my tests setting --parallel to 20 uses about 500Mi