search

datto / dattobd

kernel module for taking block-level snapshots and incremental backups of Linux block devices
GNU General Public License v2.0
569 stars 121 forks source link

Will the system memory exhaust? #261

Open zhanghaiyang9999 opened 2 years ago

zhanghaiyang9999 commented 2 years ago

After creating the snapshot, if the original volume data changes very frequently, will the system memory be exhausted? Because when one bio comes, need to clone one bio and add the bio to the queue via bio_queue_add.

If the cow file writing is slower than the original volume writing speed, the system memory must be used exhaust and cause system crash, right?

nixomose commented 2 years ago

yes, but there is a limit. a given block will only be cowed once. So for example, the superblock gets rewritten to many times, it's only going to end up in the cow file the first time. that said if you take the worse case scenario and rewrite the entire disk while taking a snapshot, yes you could run into that problem, but you'll blow out the cow file before that happens.

zhanghaiyang9999 commented 2 years ago

thanks, you are right, but maybe no need to rewitte entire disk, for example just rewrite 10G data, maybe enter the worse case? -------- 原始信息 --------发件人： nixomose @.> 日期: 2021/10/7 19:44 (GMT+08:00) 收件人： datto/dattobd @.> 抄送： zhanghaiyang @.>, Author @.> 主题： Re: [datto/dattobd] Will the system memory exhaust? (#261) yes, but there is a limit. a given block will only be cowed once. So for example, the superblock gets rewritten to many times, it's only going to end up in the cow file the first time. that said if you take the worse case scenario and rewrite the entire disk while taking a snapshot, yes you could run into that problem, but you'll blow out the cow file before that happens.

—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

[ { @.": "http://schema.org", @.": "EmailMessage", "potentialAction": { @.": "ViewAction", "target": "https://github.com/datto/dattobd/issues/261#issuecomment-937713753", "url": "https://github.com/datto/dattobd/issues/261#issuecomment-937713753", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { @.": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

nixomose commented 2 years ago

yeah. We have seen the effect you are talking about but we've never run into the case where we ran out of memory before we ran out of cow file. maybe you could contrive such a situation, but we've never seen it in normal use.