MMetelko
commented
6 years ago Choosing to table this idea for now. Here is the latest message exchange that shows exploration into the issue.
From Abhishek: Date: Saturday, January 6, 2018 at 10:36 AM
Hi Gabor
We used btrfs subvolumes for COSMOS and there were not that many issues. BTRFS also has other benefits like COW.
However, if our use case just demands quota per user then CONFIG_QUOTA should be good enough. Note that we will have to assign a user id to applications and apply quota for those user ids. Note that CONFIG_QUOTA requires creation of a quota file (to be readable/writable only by root) in the directory on which we want to apply quota.
Sent from Mail for Windows 10
From: Karsai, Gabor Sent: Saturday, January 6, 2018 9:16 AM To: riaps-vu@list.isis.vanderbilt.edu Subject: [riaps-vu] FW: btrfs quota exceeded notifications through netlink sockets
Please see below - it seems btrfs is not a good solution. (On the bbb we are using an even older version of the kernel.)
-----Original Message----- From: Duncan [mailto:1i5t5.duncan@cox.net] Sent: Saturday, January 06, 2018 3:41 AM To: Karsai, Gabor gabor.karsai@Vanderbilt.Edu Subject: Re: btrfs quota exceeded notifications through netlink sockets
[This mail was also posted to gmane.comp.file-systems.btrfs.]
Karsai, Gabor posted on Sat, 06 Jan 2018 01:34:09 +0000 as excerpted:
I created a subvolume on a btrfs, set a limit and the quota is enforced
- dumping too much data into the subvolume results in a 'quota exceeded' message (from dd, for example). But when I am trying to get netlink socket notifications, nothing arrives on the socket (I am using pyroute2 which is supposedly able to receive disk quota notifications)
$ uname -a Linux riaps-dev 4.10.0-42-generic #46~16.04.1-Ubuntu SMP Mon Dec 4 15:57:59 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
btrfs: whatever Ubuntu 16.04 has
Kconfig: CONFIG_QUOTA_NETLINK_INTERFACE=y
Someone with a bit more knowledge of quotas in general and btrfs quotas in particular will hopefully confirm (I'm just a btrfs user and list regular, without quotas in my own use-case, so this is only what I've seen on-list), but sometimes a fast non-authoritative answer can be more useful than a slower authoritative answer, so here's the first...
Btrfs quotas don't use the normal kernel quota mechanism as they work somewhat differently. Indeed, the kernel-config help for CONFIG_QUOTA doesn't mention btrfs support at all. As such, I don't believe the btrfs quota subsystem uses the normal kernel quota netlink interface at all. At least, I've never seen it mentioned, and it would surprise me if btrfs quotas /did/ use that interface, because they are different enough to be unlikely to properly match the expected interface API.
Meanwhile, be aware that until recently btrfs quotas were too buggy to be used reliably. While they work rather better now, more minor fixes are still being made, with every recent kernel including 4.14 having quota fixes. For this feature a current kernel is definitely recommended, and 4.10 is neither an LTS kernel series (4.9 and 4.14 are the two most recent and best supported LTS series, 4.10 was simply a normal kernel and only had upstream support thru 4.11) nor within the latest two current kernels, so on-list support won't be as good as if you were running an LTS or current kernel.
And even with the fixes, enabling quotas increases btrfs scaling issues when running commands such as btrfs balance and check, tho normal runtime performance isn't so severely affected. Balance in particular takes / much/ longer when quotas are enabled due to constant quota updates as the blockgroups are moved around, so temporarily disabling them during balances is recommended to speed up the balance. Unfortunately, the scenarios under which you're likely to need to run check, when the filesystem won't mount, prevent disabling quotas then.
So while quota numbers should be reliable with supported kernels now, leaving them off unless you really need the feature is still recommended.
The one good thing for use-cases that /do/ need quotas is that such use- cases tend to be commercial systems with proper backups, where the performance of commands such as balance and check may not matter so much, since maintenance in such use-cases often consists of failing the entire filesystem and falling back to the hot-spare, rather than trying to do on- the-fly filesystem maintenance such as rebalancing to a new device or raid layout or checking and trying to repair a filesystem that won't mount. Since normal runtime performance isn't particularly affected, quotas tend to be fine for such use-cases.
-- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
When ready, release the BTRFS version of the BBB image.