Unclear documentation about exclusion patterns

[christophehenry]

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

Issue on documentatioon

Following up borgbase/vorta#1487, I'm raising this issue to point that Borg's documentation on patterns can be confusing. It specifies:

If you give /absolute/ as root, the paths going into the matcher will look relative like absolute/…/file.ext, because file paths in Borg archives are always stored normalized and relative.

What I understood from that sentence, at first, is if you provide an absolute path to backup, the exlusion patterns will be matched relative to the provided path.

So, backing up /home/user and trying to exclude /home/user/directory-to-exclude, I tried a bunch of things like:

directory-to-exclude
/directory-to-exclude
pp:directory-to-exclude
# …

that didn't work.

That's only when the Vorta team explained that, when provided an absolute path, the backup path is considered relative to / so the exclusion patterns will be matched against /, not the whole backup path — which makes sense since multiple backup path can be supplied.

So maybe the docs can be reworded to disambiguate them. It should probably focus less on how Borg processes these path internally (the whole "file paths in Borg archives are always stored normalized and relative") and more what patterns should I actually supply for absolute and relative paths" (most examples focus on / which does not remove the ambiguity).

[ThomasWaldmann]

Well, maybe just read the sentence you quoted again. It gives you a specific example how the archived path that is used for matching looks and i can't see how one can come to a different understanding after that.

The patterns you need to supply just need to match that archived path, no more, no less.

[ThomasWaldmann]

Maybe we could additionally provide an example that uses a recursion root like ./relative.

[christophehenry]

i can't see how one can come to a different understanding after that

Well… One can since… I have. Look, if I couldn't clearly understand that part of the docs, I'm probably not alone even if I'm the first to report it. I'm just trying to help. Even though english is not my mother tongue, I understand it quite correclty and that part of the docs was obscure to me so it most likely is to others.

[ThomasWaldmann]

OK, so which specific part of that sentence led you to wrong conclusions?

Would it be better if we remove the "because file paths in Borg archives are always stored normalized and relative" part?

[real-yfprojects]

Maybe we could additionally provide an example that uses a recursion root like ./relative.

Yeah. I think, the docs supply all the relevant information besides this one although they do not state it in a clear and structured way. The first paragraph seems to contradict the second one in regards to relative paths. Also the normalizing could be elaborated since ../path is stored the same way as ./path.

[christophehenry]

There's also a problem with the first paragraphe which uses … (ellipsis) in the path. If you have vision problemsor if you read the sentence too quickly you may mistake with .. (double-dot) or you may think it's a typo. This makes the whole paragraph about how path are stored by Borg a bit obscure.

Maybe this paragraph is superfluous? If I'm seeking informations about how to exclude files and directories from backup, I don't think I necessarily need to understand the technical aspects of how Borg stores the paths. I just need to know that, if I provide a relative path, the files are matched against ./ and if I provide an absolute path, the files are matched against / and not the whole /home/user backup path like I thought it did. And maybe add one or more examples with borg create -e home/user/directory-to-exclude backup /home/user/?

Edit: oh, pardon me, there is indeed the example:

borg create -e home/user/cache/ backup / /home/user/cache/important

But it covers another situation and doesn't totally disambiguate the docs since / is still being backed up in the example.

[real-yfprojects]

Also the patterns /home/user/some/path and home/user/some/path work correctly, although the former isn't documented.

[ThomasWaldmann]

@real-yfprojects to avoid breakage, some borg versions removed a leading slash from simple patterns. For more complex pattern styles (like e.g. regex) that is impossible to support though, so this "leading slash removal" was not doable in a consistent way.

IIRC, starting from 1.2.x, patterns must not try to match a leading slash, it is just not there in the archived paths and thus can never match.

[real-yfprojects]

IIRC, starting from 1.2.x, patterns must not try to match a leading slash, it is just not there in the archived paths and thus can never match.

I should update my patterns then.

[edsonsbj]

Hi! Could any of you help me? I’m having the following problem: when trying to back up my Plex server, it ends up copying everything, ignoring the folders that I don’t want it to back up.

P sh 
R / 
+ var/snap/plexmediaserver/common/Library/Application Support/Plex Media Server
- var/snap/plexmediaserver/common/Library/Application Support/Plex Media Server/Cache
- var/snap/plexmediaserver/common/Library/Application Support/Plex Media Server/Crash Reports
- var/snap/plexmediaserver/common/Library/Application Support/Plex Media Server/Diagnostics
- var/snap/plexmediaserver/common/Library/Application Support/Plex Media Server/Logs

[real-yfprojects]

Borg goes through the pattern file from top to bottom. With var/snap/plexmediaserver/common/Library/Application Support/Plex Media Server it first hits the rule to include that path so it doesn't look any further down in the pattern file. If you move the line starting with + down to the bottom your file should work.

[edsonsbj]

Borg goes through the pattern file from top to bottom. With var/snap/plexmediaserver/common/Library/Application Support/Plex Media Server it first hits the rule to include that path so it doesn't look any further down in the pattern file. If you move the line starting with + down to the bottom your file should work.

Thank you very much for the answer I've been trying to solve it for 2 days and now it worked.

[IMarvinTPA]

Can the documentation have an example of how to exclude a folder pattern of "steamapps/common" anywhere in the path? IE, a simple example showing what is expected if you don't care where in the path the folder+subfolder may appear. My first guess of just "steamapps/common" did not work. The documentation reads that it should be doing basically a substring search of the path for the pattern you give unless you add stars or start with a slash.