Open dajhorn opened 11 years ago
This is great! Slow snapshot listings (really the high load associated with) have been my #1 headache with running frequent snapshots (which are super-handy).
System #1:
# time zfs list -H -t snapshot -S creation -o name | wc -l
1686
real 0m32.509s
user 0m0.208s
sys 0m0.624s
# time zfs list -H -t snapshot -o name -s name|grep zfs-auto-snap |awk '{ print substr( $0, length($0) - 14, length($0) ) " " $0}' |sort -r -k1,1 -k2,2|awk '{ print substr( $0, 17, length($0) )}' | wc -l
1686
real 0m0.130s
user 0m0.048s
sys 0m0.075s
System #2:
# time zfs list -H -t snapshot -S creation -o name | wc -l
3755
real 5m8.268s
user 0m1.039s
sys 0m13.140s
# time zfs list -H -t snapshot -o name -s name|grep zfs-auto-snap |awk '{ print substr( $0, length($0) - 14, length($0) ) " " $0}' |sort -r -k1,1 -k2,2|awk '{ print substr( $0, 17, length($0) )}' | wc -l
3755
real 0m4.709s
user 0m0.237s
sys 0m0.845s
I've edited my crontabs and no errors so far (just a few frequents of testing), but I can't imagine switching back unless something unexpected happens.
further performance optimization is possible, if zfs-auto-snapshot would not pull a whole list of all snapshots but only the list of the appropriate filesystem (or sub-fs) to be snapshotted. that may (depending of use case) skip most of the snapshots ( see https://github.com/zfsonlinux/zfs-auto-snapshot/issues/62 )
pull whole list of ALL snapshots
real 7m49.567s
pull whole list in a more efficient way (--fast option)
real 0m3.978s real 0m4.359s real 0m6.028s real 0m6.199s real 0m5.385s real 0m8.575s
pull only the sub-fs the traditional way. mind that this also speeds up things a LOT if you have a lot of snapshots
real 0m4.984s real 0m5.084s real 0m5.631s real 0m6.671s real 0m8.397s real 0m5.398s
(fastest) pull only sub-fs in a more efficient way (let`s combine --fast + pull only sub-fs list)
real 0m0.210s real 0m0.058s real 0m0.121s real 0m0.111s real 0m0.024s real 0m0.125s
Pursuant to zfsonlinux/zfs#450, the slow return of some
zfs list
invocations is arguably a bug. The merge in zfsonlinux/zfs-auto-snapshot#15 kludges thezfs list
invocation to improve performance.Provide a switch to allow the user to choose between correctness or performance. If corner-cases are discovered in the kludge, then add a
--fast
switch. Otherwise, add a--slow
switch to preserve the original code until performance improves.