search

rockstor / rockstor-core

Linux/BTRFS based Network Attached Storage(NAS)
http://rockstor.com/docs/contribute_section.html
GNU General Public License v3.0
558 stars 138 forks source link

Quota file exists error on Share resize #2854 #2874

Closed phillxnet closed 4 months ago

phillxnet commented 4 months ago

Fix insensitivity re Share quota id membership of Pool associated quote group, when the Share's quota id has multiple parent quota groups. Avoiding consequent failure during redundant quota group assignment.

Includes:

Fixes #2854

phillxnet commented 4 months ago

Testing

We have a new unit-test with this PR:

cd /opt/rockstor/src/rockstor/fs
poetry run django-admin test -v 3 -p test_btrfs*
...
test_qgroup_is_assigned (rockstor.fs.tests.test_btrfs.BTRFSTests.test_qgroup_is_assigned)
Old and new btrfs-progs output for "BTRFS qgroup show -pc mnt_pt" ... ok
...

With this draft PR's branch we no longer see the issue detailed error, and if share resize is within Pool size, Resize requests are handled correctly (noting that they are currently only cosmetic): however if a Share Resize request exceeds its Parent Pool size we fail silently on the Web-UI and maintain the prior Share size.

Work in progress/draft status for now.

phillxnet commented 4 months ago

Testing continued

An rpm was built using this branch (Leap 15.6 X86_64 host) and a prior pool was imported (a requirement to reproduce the issue this PR addresses). Two imported shares on this pool both allowed for Resize within the limits of their parent Pool. Both shares were confirmed to have multiple qgroup parents:

rleap15-6:~ # btrfs qgroup show -pc /mnt2/rock-pool
Qgroupid    Referenced    Exclusive Parent                                                                  Child   Path 
--------    ----------    --------- ------                                                                  -----   ---- 
0/5           16.00KiB     16.00KiB -                                                                       -       <toplevel>
0/268         16.00KiB     16.00KiB 2015/75,2015/76,2015/77,2015/78,2015/79,2015/80,2015/81,2015/82,2015/84 -       sftp-share
0/269         16.00KiB     16.00KiB 2015/83,2015/85                                                         -       size-test
2015/1           0.00B        0.00B -                                                                       -       <0 member qgroups>
2015/2           0.00B        0.00B -                                                                       -       <0 member qgroups>
2015/3           0.00B        0.00B -                                                                       -       <0 member qgroups>
...
2015/74          0.00B        0.00B -                                                                       -       <0 member qgroups>
2015/75       16.00KiB     16.00KiB -                                                                       0/268   <0 member qgroups>
2015/76       16.00KiB     16.00KiB -                                                                       0/268   <0 member qgroups>
2015/77       16.00KiB     16.00KiB -                                                                       0/268   <0 member qgroups>
2015/78       16.00KiB     16.00KiB -                                                                       0/268   <0 member qgroups>
2015/79       16.00KiB     16.00KiB -                                                                       0/268   <0 member qgroups>
2015/80       16.00KiB     16.00KiB -                                                                       0/268   <0 member qgroups>
2015/81       16.00KiB     16.00KiB -                                                                       0/268   <0 member qgroups>
2015/82       16.00KiB     16.00KiB -                                                                       0/268   <0 member qgroups>
2015/83       16.00KiB     16.00KiB -                                                                       0/269   <0 member qgroups>
2015/84       16.00KiB     16.00KiB -                                                                       0/268   <0 member qgroups>
2015/85       16.00KiB     16.00KiB -                                                                       0/269   <0 member qgroups>

A fresh share was then created (Web-UI) at the default 1 GB size:

Qgroupid    Referenced    Exclusive Parent                                                                  Child   Path 
--------    ----------    --------- ------                                                                  -----   ---- 
...
0/270         16.00KiB     16.00KiB 2015/86                                                                 -       pr2874test
...
2015/86       16.00KiB     16.00KiB -                                                                       0/270   <0 member qgroups>

And this newly created Share (single Rockstor created parent qgroup as not imported) was also amenable to Share resize within Pool limits.

Quota status cycle

btrfs quota disable /mnt2/rock-pool

In Quotas Disabled (Web-UI surfaced) we were also able to accomplish Share Resize.

rleap15-6:~ # btrfs qgroup show -pc /mnt2/rock-pool
ERROR: can't list qgroups: quotas not enabled

And after a Web-UI quote enable (Pool details page), we can also still resize Shares: note however that this quota cycling clears the existing 'tally' of Share (subvol) and rockstor created quota groups. I.e. our prior pool with considerable quota history (many sequential imports) now shows as:

rleap15-6:~ # btrfs qgroup show -pc /mnt2/rock-pool
Qgroupid    Referenced    Exclusive Parent   Child   Path 
--------    ----------    --------- ------   -----   ---- 
0/5           16.00KiB     16.00KiB -        -       <toplevel>
0/268         16.00KiB     16.00KiB 2015/1   -       sftp-share
0/269         16.00KiB     16.00KiB 2015/2   -       size-test
0/270         16.00KiB     16.00KiB 2015/3   -       pr2874test
2015/1        16.00KiB     16.00KiB -        0/268   <0 member qgroups>
2015/2        16.00KiB     16.00KiB -        0/269   <0 member qgroups>
2015/3        16.00KiB     16.00KiB -        0/270   <0 member qgroups>

That-is akin to a freshly created Pool (quota wise). Where each share has only a single Rockstor assigned (2015/*) parent qgroup.

phillxnet commented 4 months ago

Re-import of prior test Pool

A hard-reset was performed (DB wipe) via:

systemctl stop rockstor*
rm /opt/rockstor/.initrock
userdel radmin  # Web-UI admin user established during setup

as per developer docs entry: https://rockstor.com/docs/contribute/contribute.html#code-test

And the same pool (post quote cli disable / Web-UI enable) sequence in the last comment here; then indicates (afer Web-UI import) the following qouta group arrangements:

rleap15-6:~ # btrfs qgroup show -pc /mnt2/rock-pool
Qgroupid    Referenced    Exclusive Parent        Child   Path 
--------    ----------    --------- ------        -----   ---- 
0/5           16.00KiB     16.00KiB -             -       <toplevel>
0/268         16.00KiB     16.00KiB 2015/1,2015/4 -       sftp-share
0/269         16.00KiB     16.00KiB 2015/2,2015/5 -       size-test
0/270         16.00KiB     16.00KiB 2015/3,2015/6 -       pr2874test
2015/1        16.00KiB     16.00KiB -             0/268   <0 member qgroups>
2015/2        16.00KiB     16.00KiB -             0/269   <0 member qgroups>
2015/3        16.00KiB     16.00KiB -             0/270   <0 member qgroups>
2015/4        16.00KiB     16.00KiB -             0/268   <0 member qgroups>
2015/5        16.00KiB     16.00KiB -             0/269   <0 member qgroups>
2015/6        16.00KiB     16.00KiB -             0/270   <0 member qgroups>

Indicating our first Rockstor assigned parent doubling affect re parent qgroups. All 3 shares now have 2 parents each. This multi-parent blindness was why our Share Resize tried to create quote groups to manage space usage that actually already existed: ergo issue detailed:

‘ERROR: unable to assign quota group: File exists’

Where the 'file' is actually the quota group required for usage management/reporting.

phillxnet commented 4 months ago

Based on the indicated testing I think we have at least a fix for the indicated issue here. I'll merge ready to move to getting this and other unreleased changes out via another testing rpm.

There are definite refinements to be done in our quota handling/management and this will hopefully be a new start in improvements in this area. It's been a long-time coming but there is renewed interest, and a lot of upstream improvements, in quota support. Plus our quota banners are a bad show and have been in place for far too long now.