search

denshoproject / ddr-cmdln

Command-line tools for automating the Densho Digital Repository's various processes.
Other
0 stars 2 forks source link

`Permission denied` when re-uploading an existing file #196

Closed gjost closed 3 years ago

gjost commented 3 years ago

When user re-uploaded a previously uploaded file, they see the following error:

Traceback (most recent call last):
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
    return self.run(*args, **kwargs)
  File "/opt/ddr-local/ddrlocal/webui/tasks/files.py", line 189, in file_add_local
    git_name, git_mail, agent=settings.AGENT
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/ddr_cmdln-5.0.16-py3.7.egg/DDR/models/entity.py", line 657, in add_local_file
    return ingest.add_file(data, self, git_name, git_mail, agent)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/ddr_cmdln-5.0.16-py3.7.egg/DDR/ingest.py", line 165, in add_file
    copy_to_file_path(file_, src_path, log)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/ddr_cmdln-5.0.16-py3.7.egg/DDR/ingest.py", line 307, in copy_to_file_path
    shutil.copy(src_path, file_.path_abs)
  File "/usr/lib/python3.7/shutil.py", line 245, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.7/shutil.py", line 121, in copyfile
    with open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: '/var/www/media/ddr/ddr-densho-421/files/ddr-densho-421-1/files/ddr-densho-421-1-master-7a264e9c53.tif'

Instead of this ugly error they should see a nice "Already uploaded this file to $ENTITY with role $ROLE." message.

gjost commented 3 years ago

It doesn't make sense that there should be a PermissionError when the repo and files dirs were created by the same process.

Problem occurs when re-uploading an existing file. You'd think this would return a SameFileError but it's not the same file -- the binary was previously ingested into git-annex so the actual file is a symlink pointing to the annex dir.

According to (the Python docs)[https://docs.python.org/3/library/exceptions.html?highlight=permissionerror#PermissionError], PermissionError corresponds to errno EACCES and EPERM. errno.EACCES is "Permission denied" while errno.EPERM is "Operation not permitted".

We need to either detect that the file already exists or use shutil.copyfile's follow_symlinks flag to overwrite the file in the annex dir. Will try the former.

GeoffFroh commented 3 years ago

Somewhat of an edge-case. Re-verify that the behavior still exists; if so, we'll decide whether to fix or ignore

GeoffFroh commented 3 years ago

@pkikawa will check if behavior still exists

pkikawa commented 3 years ago

behavior no longer present as of 5.2.2