search

borgbackup / borg

Deduplicating archiver with compression and authenticated encryption.
https://www.borgbackup.org/
Other
10.95k stars 740 forks source link

AssertionError & PermissionError #3894

Closed hubitor closed 6 years ago

hubitor commented 6 years ago

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

BUG

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

borg 1.1.6

Operating system (distribution) and version.

Arch Linux

Hardware / network configuration, and filesystems used.

How much data is handled by borg?

350 GB

Full borg commandline that lead to the problem (leave away excludes and passwords)

borg list /mybackups/Borg/home/

Describe the problem you're observing.

I've installed borgmatic today and tried it for the first time. It has created a backup without error messages. After that I wanted to list my archives. I normally don't use sudo for this. But I got an AssertionError and PermissionError from Borg as a normal user. If use sudo I can list my archives without any errors. To me it makes sense that Borg complained with PermissionError since apparently borgmatic created the last archive as a root user but the AssertionError doesn't.

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

Include any warning/errors/backtraces from the system logs

$ borg list /mybackups/Borg/home/ 
Exception ignored in: <bound method Repository.__del__ of <Repository /mybackups/Borg/home>>
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 174, in __del__
    assert False, "cleanup happened in Repository.__del__"
AssertionError: cleanup happened in Repository.__del__
Local Exception
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1266, in get_fd
    return self.fds[segment]
  File "/usr/lib/python3.6/site-packages/borg/lrucache.py", line 21, in __getitem__
    value = self._cache[key]  # raise KeyError if not found
KeyError: 50873

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 4312, in main
    exit_code = archiver.run(args)
  File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 4244, in run
    return set_ec(func(args))
  File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 137, in wrapper
    with repository:
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 184, in __enter__
    self.open(self.path, bool(self.exclusive), lock_wait=self.lock_wait, lock=self.do_lock)
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 406, in open
    if segment is not None and self.io.get_segment_magic(segment) == ATTIC_MAGIC:
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1298, in get_segment_magic
    fd = self.get_fd(segment)
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1268, in get_fd
    fd = open(self.segment_filename(segment), 'rb')
PermissionError: [Errno 13] Permission denied: '/mybackups/Borg/home/data/5/50873'

Platform: Linux hostess 4.16.13-2-ARCH #1 SMP PREEMPT Fri Jun 1 18:46:11 UTC 2018 x86_64
Linux: arch Arch Linux
Borg: 1.1.6  Python: CPython 3.6.5
PID: 2711  CWD: /home/hose
sys.argv: ['/usr/sbin/borg', 'list', '/mybackups/Borg/home/']
SSH_ORIGINAL_COMMAND: None
ThomasWaldmann commented 6 years ago

You likely have a permissions issue. Always use same user to access the repo.

On June 12, 2018 4:12:00 PM GMT+02:00, hubitor notifications@github.com wrote:

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

BUG

System information. For client/server mode post info for both

machines.

Your borg version (borg -V).

borg 1.1.6

Operating system (distribution) and version.

Arch Linux

Hardware / network configuration, and filesystems used.

How much data is handled by borg?

350 GB

Full borg commandline that lead to the problem (leave away

excludes and passwords) borg list /mnt/BACKUPS/Borg/home/

Describe the problem you're observing.

I've installed borgmatic today and tried it for the first time. It has created a backup without error messages. After that I wanted to list my archives. I normally don't use sudo for this. But I got an AssertionError and PermissionError from Borg as a normal user. If use sudo I can list my archives without any errors. To me it makes sense that Borg complained with PermissionError since apparently borgmatic created the last archive as a root user but the AssertionError doesn't.

Can you reproduce the problem? If so, describe how. If not,

describe troubleshooting steps you took before opening the issue.

Include any warning/errors/backtraces from the system logs

$ borg list /mybackups/Borg/home/ Exception ignored in: <bound method Repository.del of <Repository /mybackups/Borg/home>> Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/borg/repository.py", line 174, in del assert False, "cleanup happened in Repository.del" AssertionError: cleanup happened in Repository.del Local Exception Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1266, in get_fd return self.fds[segment] File "/usr/lib/python3.6/site-packages/borg/lrucache.py", line 21, in getitem value = self._cache[key] # raise KeyError if not found KeyError: 50873

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 4312, in main exit_code = archiver.run(args) File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 4244, in run return set_ec(func(args)) File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 137, in wrapper with repository: File "/usr/lib/python3.6/site-packages/borg/repository.py", line 184, in enter self.open(self.path, bool(self.exclusive), lock_wait=self.lock_wait, lock=self.do_lock) File "/usr/lib/python3.6/site-packages/borg/repository.py", line 406, in open if segment is not None and self.io.get_segment_magic(segment) == ATTIC_MAGIC: File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1298, in get_segment_magic fd = self.get_fd(segment) File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1268, in get_fd fd = open(self.segment_filename(segment), 'rb') PermissionError: [Errno 13] Permission denied: '/mybackups/Borg/home/data/5/50873'

Platform: Linux hostess 4.16.13-2-ARCH #1 SMP PREEMPT Fri Jun 1 18:46:11 UTC 2018 x86_64 Linux: arch Arch Linux Borg: 1.1.6 Python: CPython 3.6.5 PID: 2711 CWD: /home/hose sys.argv: ['/usr/sbin/borg', 'list', '/mybackups/Borg/home/'] SSH_ORIGINAL_COMMAND: None

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/borgbackup/borg/issues/3894

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

hubitor commented 6 years ago

Well I wrote that before but this long error message doesn't seem right to me. I think borg should at least throw another message.

ThomasWaldmann commented 6 years ago

There are obviously a lot of places in borg repository code that access repo files.

If you create a mixup of permissions in the repo (some stuff root, some other user(s)), such a permission denied error can also happen earlier or later, depening on your permissions mix. so, there is no quick way to check if permissions are right as there are a lot of repo files.

Not sure if we should handle that by a very high level handler just to make it "pretty". Pretty error messages can have the problem to hide useful information (for developers).

About the AssertionError: this is because the repo object is disposed by __del__ which is not the usual way we finalize this. It is caused by the other exception here and you can ignore it.

hubitor commented 6 years ago

How can I check which user created the repo and which user created the archives? I can't figure out how the ownership of the repo has suddenly changed.

ThomasWaldmann commented 6 years ago 
ls -lR repo
hubitor commented 6 years ago

This is what I get with ls -lR repo:

ls -lR /mybackups/Borg/home/                                                                       2 

/mybackups/Borg/home/ :
total 50520
-rw------- 1 hose hose      635 Aug 21  2017 config
drwx------ 8 hose hose     4096 Jun 10 22:49 data
-rw------- 1 root root   191327 Jun 12 22:17 hints.50957
-rw------- 1 root root 51516698 Jun 12 22:17 index.50957
-rw------- 1 root root      190 Jun 12 22:17 integrity.50957
-rw------- 1 root root       16 Jun 12 22:55 nonce
-rw------- 1 hose hose       26 Aug 21  2017 README

/mybackups/Borg/home/ data:
total 1168
drwx------ 2 hose hose 155648 Aug 21  2017 0
drwx------ 2 hose hose 253952 Aug 21  2017 1
drwx------ 2 hose hose 245760 Aug 21  2017 2
drwx------ 2 hose hose 253952 Jun 10 23:24 3
drwx------ 2 hose hose 253952 Jun 10 23:22 4
drwx------ 2 hose hose  16384 Jun 12 22:17 5

/mybackups/Borg/home/ data/0:
total 63823384
-rw------- 1 hose hose  5461375 Aug 21  2017 1
-rw------- 1 hose hose  5794328 Aug 21  2017 10
-rw------- 1 hose hose  5254470 Aug 21  2017 100
-rw------- 1 hose hose 11387833 Aug 21  2017 1000

...

/mybackups/Borg/home/ data/5:
total 2473912
-rw------- 1 hose hose       17 Jun 10 22:54 50316

...

-rw------- 1 hose hose  5242921 Jun 11 16:09 50757
-rw------- 1 hose hose       17 Jun 11 16:09 50758
-rw------- 1 hose hose       17 Jun 11 16:09 50760
-rw------- 1 hose hose       17 Jun 11 16:10 50762
-rw------- 1 root root       17 Jun 12 11:48 50764
-rw------- 1 root root       17 Jun 12 11:48 50766
-rw------- 1 root root       17 Jun 12 11:48 50768

And also from the backup files it`s obvious that the ownership has changed from user hose to user root on 12.06.2018. The question now is what are my options since it has been mentioned multiple time that it is not good to have mixed repo ownership and not root ownership. Should I delete the whole repo and create a new one with a normal user or is it sufficient to delete the archives user root created?

ThomasWaldmann commented 6 years ago

If the check succeeds then I guess there is no reason to start from scratch.

hubitor commented 6 years ago

I've done this:

chown -R thatuser.thatgroup REPO
borg check REPO

and there is no output for borg check. Does this mean is everything OK? When I create a backup there are no errors but the exit code is 1. What does this mean? If indeed something is wrong how can I get more debug info?

ThomasWaldmann commented 6 years ago

rc 1 means warning and there should be a warning in the output.

hubitor commented 6 years ago

Sorry I don't understand. borg check shouldn't return a message for error or not? In my case where there is no output what is it?

On the other hand borg create seems to be successful but exit code is 1. There is no warning.

ThomasWaldmann commented 6 years ago

as this is a separate issue and looks like unrelated to this issue, please open a new one and give the commands how to reproduce the issue, also borg version etc.