Copy file on backend

[frankenjoe]

Relates to #177

Adds backend.Base.copy_file() to copy a file on the backend. A default implementation is provided, which downloads the files to a temporary directory and afterward uploads it to the new location. The behavior can be overwritten by implementing a custom _copy_file() method. In case of a versioned interface, all versions of file are copied unless a specific version is selected.

Documentation

API

backend.Base / backend.Artifactory / backend.FileSystem

interface.Unversioned

interface.Versioned

Usage

...

Developer

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (4e1068d) 0.0% compared to head (df77567) 100.0%. Report is 1 commits behind head on main.

Additional details and impacted files

| [Files](https://app.codecov.io/gh/audeering/audbackend/pull/181?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=audeering) | Coverage Δ | | |---|---|---| | [audbackend/core/backend/artifactory.py](https://app.codecov.io/gh/audeering/audbackend/pull/181?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=audeering#diff-YXVkYmFja2VuZC9jb3JlL2JhY2tlbmQvYXJ0aWZhY3RvcnkucHk=) | `100.0% <100.0%> (ø)` | | | [audbackend/core/backend/base.py](https://app.codecov.io/gh/audeering/audbackend/pull/181?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=audeering#diff-YXVkYmFja2VuZC9jb3JlL2JhY2tlbmQvYmFzZS5weQ==) | `100.0% <100.0%> (ø)` | | | [audbackend/core/backend/filesystem.py](https://app.codecov.io/gh/audeering/audbackend/pull/181?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=audeering#diff-YXVkYmFja2VuZC9jb3JlL2JhY2tlbmQvZmlsZXN5c3RlbS5weQ==) | `100.0% <100.0%> (ø)` | | | [audbackend/core/interface/unversioned.py](https://app.codecov.io/gh/audeering/audbackend/pull/181?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=audeering#diff-YXVkYmFja2VuZC9jb3JlL2ludGVyZmFjZS91bnZlcnNpb25lZC5weQ==) | `100.0% <100.0%> (ø)` | | | [audbackend/core/interface/versioned.py](https://app.codecov.io/gh/audeering/audbackend/pull/181?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=audeering#diff-YXVkYmFja2VuZC9jb3JlL2ludGVyZmFjZS92ZXJzaW9uZWQucHk=) | `100.0% <100.0%> (ø)` | | ... and [9 files with indirect coverage changes](https://app.codecov.io/gh/audeering/audbackend/pull/181/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=audeering)

[frankenjoe]

BTW, what happens if a sub path exists on the backend with the same name as the file. I guess trying to overwrite it will raise an error? Would be nice to add a test for this case.

Mhh, interesting question. I guess it depends on the backend. E.g. on FileSystem if you have /sub/file.txt and then you try to create a file /sub it will fail. But on other backends, e.g. the SQL database in the developer section, it will work.

[frankenjoe]

E.g. on FileSystem if you have /sub/file.txt and then you try to create a file /sub it will fail.

Actually, it not only depends on the Backend but also the Interface. E.g. with Versioned it should even work on FileSystem since /sub/file.txt is translated to /sub/1.0.0/file.txt and /sub to /1.0.0/sub.

[hagenw]

BTW, what happens if a sub path exists on the backend with the same name as the file. I guess trying to overwrite it will raise an error? Would be nice to add a test for this case.

Mhh, interesting question. I guess it depends on the backend. E.g. on FileSystem if you have /sub/file.txt and then you try to create a file /sub it will fail. But on other backends, e.g. the SQL database in the developer section, it will work.

The case that it might fail is covered by the BackendError in the docstring then I guess?

[frankenjoe]

The case that it might fail is covered by the BackendError in the docstring then I guess?

I was expecting so, but turns out it does not :) This is because FileSystem uses shutil.copy() under the hood, which inteprets /sub as a folder if it exists and in that case creates a file with the name of the source path in that folder. Here is a minimal example that reveals this behavior:

import audeer

import audbackend

with tempfile.TemporaryDirectory() as tmp:

    path = audeer.touch(tmp, 'file.txt')
    backend = audbackend.backend.FileSystem(tmp, 'repo')
    backend.put_file(path, '/sub/file.txt')
    path = audeer.touch(tmp, 'file2.txt')
    backend.put_file(path, '/sub')
    backend.ls()

['/sub/file.txt', '/sub/file2.txt']

[frankenjoe]

But I suggest we discuss this in a separate issue, as it is not directly related to this PR.

[hagenw]

OK, I created https://github.com/audeering/audbackend/issues/184.