cthoyt / zenodo-client

A tool for automated uploading and version management of scientific data to Zenodo
MIT License
31 stars 5 forks source link

Zenodo Client

Tests PyPI PyPI - Python Version PyPI - License Documentation Status Codecov status Cookiecutter template from @cthoyt Code style: black Contributor Covenant DOI

A wrapper for the Zenodo API.

πŸ’ͺ Getting Started

The first example shows how you can set some configuration then never worry about whether it's been uploaded already or not - all baked in with pystow. On the first time this script is run, the new deposition is made, published, and the identifier is stored with the given key in your ~/.config/zenodo.ini. Next time it's run, the deposition will be looked up, and the data will be uploaded. Versioning is given automatically by date, and if multiple versions are uploaded on one day, then a dash and the revision are appended.

from zenodo_client import Creator, Metadata, ensure_zenodo

# Define the metadata that will be used on initial upload
data = Metadata(
    title='Test Upload 3',
    upload_type='dataset',
    description='test description',
    creators=[
        Creator(
            name='Hoyt, Charles Tapley',
            affiliation='Harvard Medical School',
            orcid='0000-0003-4423-4370',
        ),
    ],
)
res = ensure_zenodo(
    key='test3',  # this is a unique key you pick that will be used to store
                  # the numeric deposition ID on your local system's cache
    data=data,
    paths=[
        '/Users/cthoyt/Desktop/test1.png',
    ],
    sandbox=True,  # remove this when you're ready to upload to real Zenodo
)
from pprint import pprint

pprint(res.json())

A real-world example can be found here: https://github.com/cthoyt/nsockg.

The following example shows how to use the Zenodo uploader if you already know what your deposition identifier is.

from zenodo_client import update_zenodo

# The ID from your deposition
SANDBOX_DEP_ID = '724868'

# Paths to local files. Good to use in combination with resources that are always
# dumped to the same place by a given script
paths = [
    # os.path.join(DATABASE_DIRECTORY, 'alts_sample.tsv')
    '/Users/cthoyt/Desktop/alts_sample.tsv',
]

# Don't forget to set the ZENODO_API_TOKEN environment variable or
# any valid way to get zenodo/api_token from PyStow.
update_zenodo(SANDBOX_DEP_ID, paths)

The following example shows how to look up the latest version of a record.

from zenodo_client import Zenodo

zenodo = Zenodo()
OOH_NA_NA_RECORD = '4020486'
new_record = zenodo.get_latest_record(OOH_NA_NA_RECORD)

Even further, the latest version of names.tsv.gz can be automatically downloaded to the ~/.data/zenodo/<conceptrecid>/<version>/<path> via pystow with:

from zenodo_client import Zenodo

zenodo = Zenodo()
OOH_NA_NA_RECORD = '4020486'
new_record = zenodo.download_latest(OOH_NA_NA_RECORD, 'names.tsv.gz')

A real-world example can be found here where the latest build of the Ooh Na Na nomenclature database is automatically downloaded from Zenodo, even though the PyOBO package only hardcodes the first deposition ID.

Command Line Interface

The zenodo_client command line tool is automatically installed. It can be used from the shell with the --help flag to show all subcommands:

$ zenodo_client --help

It can be run with zenodo_client <deposition ID> <path 1> ... <path N>

πŸš€ Installation

The most recent release can be installed from PyPI with:

$ pip install zenodo_client

The most recent code and data can be installed directly from GitHub with:

$ pip install git+https://github.com/cthoyt/zenodo-client.git

πŸ‘ Contributing

Contributions, whether filing an issue, making a pull request, or forking, are appreciated. See CONTRIBUTING.md for more information on getting involved.

πŸ‘‹ Attribution

βš–οΈ License

The code in this package is licensed under the MIT License.

πŸͺ Cookiecutter

This package was created with @audreyfeldroy's cookiecutter package using @cthoyt's cookiecutter-snekpack template.

πŸ› οΈ For Developers

See developer instructions The final section of the README is for if you want to get involved by making a code contribution. ### Development Installation To install in development mode, use the following: ```bash $ git clone git+https://github.com/cthoyt/zenodo-client.git $ cd zenodo-client $ pip install -e . ``` ### πŸ₯Ό Testing After cloning the repository and installing `tox` with `pip install tox`, the unit tests in the `tests/` folder can be run reproducibly with: ```shell $ tox ``` Additionally, these tests are automatically re-run with each commit in a [GitHub Action](https://github.com/cthoyt/zenodo-client/actions?query=workflow%3ATests). ### πŸ“– Building the Documentation The documentation can be built locally using the following: ```shell $ git clone git+https://github.com/cthoyt/zenodo-client.git $ cd zenodo-client $ tox -e docs $ open docs/build/html/index.html ``` The documentation automatically installs the package as well as the `docs` extra specified in the [`setup.cfg`](setup.cfg). `sphinx` plugins like `texext` can be added there. Additionally, they need to be added to the `extensions` list in [`docs/source/conf.py`](docs/source/conf.py). ### πŸ“¦ Making a Release After installing the package in development mode and installing `tox` with `pip install tox`, the commands for making a new release are contained within the `finish` environment in `tox.ini`. Run the following from the shell: ```shell $ tox -e finish ``` This script does the following: 1. Uses [Bump2Version](https://github.com/c4urself/bump2version) to switch the version number in the `setup.cfg`, `src/zenodo_client/version.py`, and [`docs/source/conf.py`](docs/source/conf.py) to not have the `-dev` suffix 2. Packages the code in both a tar archive and a wheel using [`build`](https://github.com/pypa/build) 3. Uploads to PyPI using [`twine`](https://github.com/pypa/twine). Be sure to have a `.pypirc` file configured to avoid the need for manual input at this step 4. Push to GitHub. You'll need to make a release going with the commit where the version was bumped. 5. Bump the version to the next patch. If you made big changes and want to bump the version by minor, you can use `tox -e bumpversion -- minor` after.