xcp-ng / xcp

Entry point for issues and wiki. Also contains some scripts and sources.
https://xcp-ng.org
1.31k stars 74 forks source link

Feature request: Dom0 should be mounting EXT filesystems with noatime (or at least relatime) #124

Open sammcj opened 5 years ago

sammcj commented 5 years ago

ext3/4 filesystems should be mounted with noatime to reduce disk IO and improve performance, infact many modern Linux distributions have made relatime the default, however that is incredibly conservative as essentially no software uses atime for operations anymore, in the 'old days' atime was used by software like mail servers to determine message state.

It has been argued that noatime should be the default mount option in the Linux kernel and there is plenty of easy to find discussion on the internet around this and the performance improvements, one example: https://administratosphere.wordpress.com/2011/07/29/instant-10-20-boost-in-disk-performance-the-noatime-option/

However, if someone can think of a good reason that XCP-ng is special and should have atime would suggest at least mounting ext3/4 filesystems with relatime.

"relative atime only updates the atime if the previous atime is older than the mtime or ctime. Like noatime, but useful for applications like mutt that need to know when a file has been read since it was last modified."


default /etc/fstab:

LABEL=root-mxdedt    /         ext3     defaults   1  1
LABEL=swap-mxdedt          swap      swap   defaults   0  0
LABEL=logs-mxdedt    /var/log         ext3     defaults   0  2
/opt/xensource/packages/iso/XenCenter.iso   /var/xen/xc-install   iso9660   loop,ro   0

proposed /etc/fstab:

LABEL=root-mxdedt    /         ext3     defaults,noatime   1  1
LABEL=swap-mxdedt          swap      swap   defaults   0  0
LABEL=logs-mxdedt    /var/log         ext3     defaults,noatime  0  2
/opt/xensource/packages/iso/XenCenter.iso   /var/xen/xc-install   iso9660   loop,ro   0

This is an improvement suggestion as per a brief discussion on the XCP-ng forums based on my XenServer and CentOS tuning experiences.

sammcj commented 5 years ago

Note: We've been running XenServer dom0 with the ext3 filesystems mounted as ext4 with no atime for 3 or so years without issue across 32~ physical hosts and thousands of VMs.

When we first install XenServer (now XCP-ng) we change fstab as such:

sed -i 's/\/         ext3     defaults /\/         ext4     defaults,noatime/g' /etc/fstab
sed -i 's/\/var\/log         ext3     defaults /\/var\/log         ext4     defaults,noatime/g' /etc/fstab
gdelafond commented 5 years ago

Hello, did you make some benchmark to check if it changed something for VM hosted in a such modified host ?

stormi commented 5 years ago

Thanks for the suggestion. It's worth studying to see whether it has a measurable effect and whether there's a use case that would make atime useful in XCP-ng.

The output of mount shows that ext3 filesystems are mounted with relatime so it's possible that the performance boost of using atime would be negligible. I'd be interested in benchmark results, too.

stormi commented 5 years ago

There's also lazytime in the 4.x kernels that will cache access times in memory.

sammcj commented 5 years ago

I guess the question between relatime and noatime is: are you really using any software that requires atime?

stormi commented 5 years ago

Until we master 100% of XenServer's codebase, to me the question is rather: does noatime bring significant performance benefits over relatime?

sammcj commented 5 years ago

You mean noatime?

Probably not over relatime, but considering that xcp-ng still uses ext3 and has massive inefficiencies in tapdisk etc... IO performance at the moment IMHO is death by a thousand paper cuts.

Should it be a priority over other performance improvements? That’s not at all for me to decided, but I do know that many small easy wins (if safe) can help offset the harder, longer-term big wins.

-- Sam McLeod

On 7 Jan 2019, at 20:33, Samuel VERSCHELDE notifications@github.com wrote:

Until we master 100% of XenServer's codebase, to me the question is rather: does atime bring significant performance benefits over relatime?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

stormi commented 5 years ago

Yes, I meant noatime and fixed the comment :)

rushikeshjadhav commented 5 years ago

Even though fstab entry says ext3, XCP mounts it using ext4 subsystem

Dmesg says [ 28.196942] EXT4-fs (dm-0): mounting ext3 file system using the ext4 subsystem [ 28.220010] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null)

So I'm not sure how much of performance gain can be there by formatting/mounting it as ext4 via fstab. Lets see if someone can contribute some performance benchmarks.

I'd prefer XFS over EXT3/4 as it would require probably similar changes. But will add more functions including snapshot. XFS works fine as Local SR.