search

radiocosmology / alpenhorn

Alpenhorn is a service for managing an archive of scientific data.
MIT License
2 stars 1 forks source link

`PermissionError` causes global abort #176

Closed ketiltrout closed 4 months ago

ketiltrout commented 8 months ago

Probably needs to be less catastrophic...

Dec 24 06:44:15 INFO >> [Worker#1] Beginning task Check copy#2257013 on cedar_staging
Dec 24 06:44:15 ERROR >> [Worker#1] Aborting due to uncaught exception in task
Traceback (most recent call last):
  File "/home/chimedat/git/alpenhorn2/alpenhorn/pool.py", line 97, in run
    task()
  File "/home/chimedat/git/alpenhorn2/alpenhorn/task.py", line 107, in __call__
    result = self._func(self, *self._args, **self._kwargs)
  File "/home/chimedat/git/alpenhorn2/alpenhorn/io/_default_asyncs.py", line 206, in check_async
    md5sum = io.md5(fullpath)
  File "/home/chimedat/git/alpenhorn2/alpenhorn/io/default.py", line 289, in md5
    return util.md5sum_file(pathlib.Path(self.node.root, path, *segments))
  File "/home/chimedat/git/alpenhorn2/alpenhorn/util.py", line 81, in md5sum_file
    with open(filename, "rb") as f:
PermissionError: [Errno 13] Permission denied: '/project/rpp-chime/chime/chime_staging/20231214T180505Z_chimetiming_corr/00819509_0000.h5'
Dec 24 06:44:15 INFO >> [Worker#2] Stopped due to global abort.
Dec 24 06:44:15 INFO >> [Worker#3] Stopped due to global abort.
Dec 24 06:44:15 INFO >> [Worker#4] Stopped due to global abort.
Dec 24 06:44:15 INFO >> [MainThread] Node cedar_nearline: 23.30 TiB available.
Dec 24 06:44:15 INFO >> [MainThread] Updating node "cedar_nearline".
Dec 24 06:44:15 INFO >> [MainThread] Node cedar_smallfile: 9.354 PiB available.
Dec 24 06:44:15 INFO >> [MainThread] Updating node "cedar_smallfile".
Dec 24 06:44:15 INFO >> [MainThread] Updating group "cedar_offload".
Dec 24 06:44:15 INFO >> [Worker#5] Stopped due to global abort.
Dec 24 06:44:15 INFO >> [Worker#6] Stopped due to global abort.
Dec 24 06:44:16 INFO >> [MainThread] Updating group "cedar_online".
Dec 24 06:44:16 INFO >> [Worker#7] Stopped due to global abort.
Dec 24 06:44:16 INFO >> [MainThread] Updating group "cedar_staging".
Dec 24 06:44:16 INFO >> [Worker#8] Stopped due to global abort.
Dec 24 06:44:16 WARNING >> [MainThread] Skipping pull request for 20231214T180505Z_chimetiming_corr/00819509_0000.h5: existing copy in group cedar_staging needs check.
Dec 24 06:44:16 INFO >> [MainThread] Updating group "cedar_nearline".
Dec 24 06:44:17 INFO >> [MainThread] Main loop execution was 5.8s.
Dec 24 06:44:17 INFO >> [MainThread] Tasks: 28 queued, 0 deferred, 1 in-progress on 8 workers
Dec 24 06:44:17 WARNING >> [MainThread] Exiting due to global abort