huggingface / dataset-viewer

Backend that powers the dataset viewer on Hugging Face dataset pages through a public API.
https://huggingface.co/docs/dataset-viewer
Apache License 2.0
696 stars 79 forks source link

Rights error for statistics step #1583

Closed severo closed 1 year ago

severo commented 1 year ago
INFO: 2023-07-28 16:40:10,399 - root - [split-descriptive-statistics] compute JobManager(job_id=64c3ef6a6c181e70c1093bb7 dataset=Melanit/testsetneuraluma job_info={'job_id': '64c3ef6a6c181e70c1093bb7', 'type': 'split-descriptive-statistics', 'params': {'dataset': 'Melanit/testsetneuraluma', 'revision': '406baec5a6d9379698896d92041f8516ffbb6ba3', 'config': 'Melanit--testsetneuraluma', 'split': 'exampledataset'}, 'priority': <Priority.NORMAL: 'normal'>, 'difficulty': 70}
ERROR: 2023-07-28 16:40:10,401 - root - [Errno 13] Permission denied: '/stats-cache/21756284577990-split-descriptive-statistics-Melanit-testsetneura-aff1fad6'
Traceback (most recent call last):
  File "/src/services/worker/src/worker/job_manager.py", line 152, in process
    self.job_runner.pre_compute()
  File "/src/services/worker/src/worker/job_runners/_job_runner_with_cache.py", line 56, in pre_compute
    self.cache_subdirectory = Path(init_dir(new_directory))
  File "/src/libs/libcommon/src/libcommon/storage.py", line 39, in init_dir
    makedirs(directory, exist_ok=True)
  File "/usr/local/lib/python3.9/os.py", line 225, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/stats-cache/21756284577990-split-descriptive-statistics-Melanit-testsetneura-aff1fad6'
severo commented 1 year ago

we should relaunch the jobs for the steps with that error, because (normally) it's fixed in the infra.

severo commented 1 year ago

Also: it would be good to have a specific x-error-code for this error, instead of resulting in an UnexpectedError

severo commented 1 year ago

fixed by #1738