FAQ: Number of snapshots vs disk performance

digint / btrbk

Tool for creating snapshots and remote backups of btrfs subvolumes

https://digint.ch/btrbk/

GNU General Public License v3.0

1.64k stars 120 forks source link

FAQ: Number of snapshots vs disk performance #487

Open danboid opened 2 years ago

danboid commented 2 years ago

I've read the btrbk FAQ but it didn't answer my only question. How many snapshots of a subvolume does BTRFS let the user create before disk performance begins to suffer?

Last time I looked into this, online sources were telling me to avoid creating more than 20 or so snapshots per subvolume to avoid degraded performance. Is that no longer the case?

Has anyone done some decent research / tests regarding this using a recent kernel?

danboid commented 2 years ago

That info is also missing from here:

https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Subvolumes

Which would seem like the place most likely to tell me this apart from the btrbk FAQ.

digint commented 2 years ago

I think the correct answer is "it depends", and that's probably why you can't find reliable information on this.

What I can say is that my backup servers contain thousand-ish subvolumes, causing a couple of seconds mount time on a slow embedded device (>3000 subvolumes currently, not going to reboot it just to check right now ;-).

From my experience I would say:

100 subvolumes: no noticeable performance drop
1000 subvolumes: slight performance drop
10000 subvolumes: unknown, probably ok on a backup server, maybe not ok on a production server

danboid commented 2 years ago

Thanks for the useful feedback @digint !

I've heard ZFS performance starts to suffer around the 10000 snapshots mark. I've also heard bcachefs can handle millions of snapshots without any problems, which will be cool if true. I've not tried bcachefs yet.