automorphism88 / snapraid-btrfs

Script for using snapraid with btrfs snapshots
GNU General Public License v3.0
135 stars 16 forks source link

No snapper configs found #23

Open mybigman opened 2 years ago

mybigman commented 2 years ago

Not sure whats going on here or what I have missed :/

These are fresh drives no initial content.

> snapraid-btrfs list
Snapper configs found:

snapraid-btrfs: No snapper configs found for any data drives in /etc/snapraid.conf
> cat /etc/snapraid.conf
# parity
1-parity /mnt/parity1/snapraid.parity

# content
content /var/snapraid.content
content /mnt/snapraid-content/disk1/snapraid.content
content /mnt/snapraid-content/disk2/snapraid.content

# data
data d1 /mnt/disk1
data d2 /mnt/disk2

# exclude
exclude *.unrecoverable
exclude /tmp/
exclude /lost+found/
exclude downloads/
exclude appdata/
exclude *.!sync
exclude /.snapshots/
> snapper list-configs
Config        | Subvolume
--------------+-----------
mergerfsdisk1 | /mnt/disk1
mergerfsdisk2 | /mnt/disk2
> sudo btrfs subvol list /mnt/disk1
ID 256 gen 20 top level 5 path data
ID 257 gen 14 top level 5 path content
ID 258 gen 16 top level 256 path .snapshots

> sudo btrfs subvol list /mnt/disk2
ID 256 gen 19 top level 5 path data
ID 257 gen 14 top level 5 path content
ID 258 gen 16 top level 256 path .snapshots
mybigman commented 2 years ago

This will only run under root... with warnings though.

> sudo snapraid-btrfs list
snapraid-btrfs: WARNING: Running snapraid-btrfs as root is not recommended
snapraid-btrfs: WARNING: (nor is running snapraid as root)
Snapper configs found:
mergerfsdisk1 /mnt/disk1
mergerfsdisk2 /mnt/disk2
...
D34DC3N73R commented 10 months ago

@mybigman did you ever figure this out? I'm running into the same issue.

mybigman commented 10 months ago

@D34DC3N73R unfortunately not. I gave up in the end and moved on from this :/

aqxa1 commented 9 months ago

The issue is caused by a snapper regression. Revert to an older version of snapper or use git.

D34DC3N73R commented 6 months ago

@aqxa1 I now have snapper 0.11.0, libsnapper 7.4.3 and it appears that snapper get-config changed somewhat. Now when I run that command I get

$ snapper get-config
The config 'root' does not exist. Likely snapper is not configured.
See 'man snapper' for further instructions.

which causes snapraid-btrfs to fail in pretty much all aspects.

getting individual configs works

$ snapper -c data1 get-config
Key                    │ Value
───────────────────────┼────────────
ALLOW_GROUPS           │ <usergroup>
ALLOW_USERS            │ <username>
BACKGROUND_COMPARISON  │ yes
EMPTY_PRE_POST_CLEANUP │ yes
EMPTY_PRE_POST_MIN_AGE │ 1800
FREE_LIMIT             │ 0.2
FSTYPE                 │ btrfs
NUMBER_CLEANUP         │ yes
NUMBER_LIMIT           │ 50
NUMBER_LIMIT_IMPORTANT │ 10
NUMBER_MIN_AGE         │ 1800
QGROUP                 │
SPACE_LIMIT            │ 0.5
SUBVOLUME              │ /mnt/data/1
SYNC_ACL               │ no
TIMELINE_CLEANUP       │ yes
TIMELINE_CREATE        │ no
TIMELINE_LIMIT_DAILY   │ 10
TIMELINE_LIMIT_HOURLY  │ 10
TIMELINE_LIMIT_MONTHLY │ 10
TIMELINE_LIMIT_WEEKLY  │ 0
TIMELINE_LIMIT_YEARLY  │ 10
TIMELINE_MIN_AGE       │ 1800

and list-configs works

$ snapper list-configs
Config │ Subvolume
───────┼─────────────
data1  │ /mnt/data/1
data10 │ /mnt/data/10
data2  │ /mnt/data/2
data3  │ /mnt/data/3
data4  │ /mnt/data/4
data5  │ /mnt/data/5
data6  │ /mnt/data/6
data7  │ /mnt/data/7
data8  │ /mnt/data/8
data9  │ /mnt/data/9

does snapraid-btrfs have to be updated to account for the changes in snapper? Or is there possibly something wrong with my snapper config? Appreciate any insight you may have.

aqxa1 commented 6 months ago

I'm using snapper 0.10.7 and libsnapper 7.4.0, but I get the message too. But it has no negative effect on snapraid-btrfs for me (and it shouldn't, because I don't believe snapraid-btrfs touches the default root config). Probably creating a root config would workaround the warning, but I don't expect it will fix your issue.

But there could be some other issue with newer snapper/libsnapper.

D34DC3N73R commented 6 months ago

I've had snapraid-btrfs working for quite some time along with the runner script, but snapper just updated yesterday to v11 and that's when the snapraid-btrfs and the runner script started to fail.

I thought it was maybe due to snapraid-btrfs using snapper get-config to get subvolume names

For each config found, attempt to read the SUBVOLUME variable using snapper get-config. If this command fails (generally because the user is not included in ALLOW_USERS or ALLOW_GROUPS), skip the config.

$ snapraid-btrfs ls
snapraid-btrfs: No snapper configs found for any data drives in /etc/snapraid.conf
$ cat /etc/snapraid.conf
# SnapRAID configuration file

# Parity location(s)
1-parity /mnt/parity/1/snapraid.parity
2-parity /mnt/parity/2/snapraid.parity

# Content file location(s)
#content /var/snapraid.content
content /mnt/content/1/snapraid.content
content /mnt/content/2/snapraid.content
content /mnt/content/3/snapraid.content
content /mnt/content/4/snapraid.content
content /mnt/content/5/snapraid.content
content /mnt/content/6/snapraid.content
content /mnt/content/7/snapraid.content
content /mnt/content/8/snapraid.content
content /mnt/content/9/snapraid.content
content /mnt/content/10/snapraid.content

# Data disks
data data1 /mnt/data/1
data data2 /mnt/data/2
data data3 /mnt/data/3
data data4 /mnt/data/4
data data5 /mnt/data/5
data data6 /mnt/data/6
data data7 /mnt/data/7
data data8 /mnt/data/8
data data9 /mnt/data/9
data data10 /mnt/data/10

# Excludes hidden files and directories
exclude *.unrecoverable
exclude /tmp/
exclude /lost+found/
exclude downloads/
exclude appdata/
exclude *.!sync
exclude /.snapshots/
D34DC3N73R commented 6 months ago

I've edited line 690 of the btrfs script after testing out those sed commands on the output and it's fixed the issue for me.

to expand a bit, this was the output using the original code

$ snapper -c data1 get-config 2>/dev/null | sed -e '/^SUBVOLUME /!d' -e 's/^SUBVOLUME[ ]*| //'
SUBVOLUME              │ /mnt/data/1

which as you can see did not remove SUBVOLUME │ from the line which caused the match to fail. ~Instead of using literal space matching [ ]* I used the space character class matching [[:space:]]* which is a lot more forgiving.~

Edit The reason this fails is snapper changing the character from | to the following should work for old and new snapper versions.

        sed -e '/^SUBVOLUME /!d' -e 's/^SUBVOLUME[ ]*[|│] //')"
y2kdread commented 6 months ago

@D34DC3N73R - I was running into the exact same issue as you and your fixed worked perfectly for me. Thanks for putting that together!

D34DC3N73R commented 6 months ago

@y2kdread can you let me know which version of snapper / libsnapper you're running along with your OS? I'd imagine this is backward compatible, but it would be nice to know. Maybe I'll submit a pull request.

D34DC3N73R commented 6 months ago

ok, it's not spaces that are the issue. The original code looks for | but now snapper outputs which is a different character. looking at them next to each other |│

y2kdread commented 6 months ago

@y2kdread can you let me know which version of snapper / libsnapper you're running along with your OS? I'd imagine this is backward compatible, but it would be nice to know. Maybe I'll submit a pull request.

Here you go!

Operating System: Debian GNU/Linux 12 (bookworm)
Kernel: Linux 6.1.0-20-amd64

and

libsnapper                           0.11.0-1
snapper                               0.11.0-1
D34DC3N73R commented 6 months ago

PR #34 opened