search

audeering / audb

Manage audio and video databases
https://audeering.github.io/audb/
Other
23 stars 1 forks source link

Loading a database can fail if a user don't have access to one of the defined repositories #333

Closed hagenw closed 10 months ago

hagenw commented 10 months ago

E.g. loading emodb from the public repository might fail with:

>>> audb.load('emodb', version='1.4.1')
Get:   emodb v1.4.1
Cache: /home/user/audb/emodb/1.4.1/d3b62a9b
Traceback (most recent call last):
  File "/home/user/.pyenv/versions/3.9.11/lib/python3.9/site-packages/audb/core/api.py", line 268, in dependencies
    deps.load(deps_path)
  File "/home/user/.pyenv/versions/3.9.11/lib/python3.9/site-packages/audb/core/dependencies.py", line 304, in load
    raise FileNotFoundError(
FileNotFoundError: [Errno 2] No such file or directory: '/home/user/audb/emodb/1.4.1/db.pkl'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.pyenv/versions/3.9.11/lib/python3.9/site-packages/dohq_artifactory/exception.py", line 20, in raise_for_status
    response.raise_for_status()
  File "/home/user/.pyenv/versions/3.9.11/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error:  for url: https://artifactory.audeering.com/artifactory/api/storage/data-private-local/emodb/db/1.4.1/db-1.4.1.yaml

My guess is that it fails with an error when looking up the backend at https://github.com/audeering/audb/blob/ba8c0d9c3b51a33c6e0588752568b3971d217bf8/audb/core/utils.py#L73-L79.

hagenw commented 10 months ago

Reading https://audeering.github.io/audbackend/api/audbackend.Artifactory.html#exists it seems that we should change

if backend.exists(header, version):

to

if backend.exists(header, version, suppress_backend_errors=True):
hagenw commented 10 months ago

The problem happens already earlier in the code when trying to access the repo, which might fail if a user has no access rights, or only access rights to certain folders within the repo, or the repo does not exist:

https://github.com/audeering/audb/blob/ba8c0d9c3b51a33c6e0588752568b3971d217bf8/audb/core/utils.py#L75

This means using suppress_backend_errors=True in backend.exists() is not needed, but I would argue that it adds no harm and we are more on the safe side with it.

In addition, it turns out that the docstring description of backend.exists has actually a bug and does not raise a BackendError if a path does not exists.