Lycophron

Lycophron is a CLI tool to support batch uploads of records to Zenodo.

The tool supports the upload through CSV files that describe each record to be uploaded.

"The" Quickstart Guide

1. Installation

   Install from GitHub using pip:

   pip install --user "lycophron @ git+https://github.com/plazi/lycophron@main"

   Alternatively, use pipx:

   pipx install "lycophron @ git+https://github.com/plazi/lycophron@main"

2. Initalize a local project named "monkeys"

   lycophron init monkeys

   List the project contents:

   cd monkeys/
   
   tree
   .
   ├── lycophron.cfg
   ├── lycophron.db
   ├── files
   │   └── Adam_Hubert_1976.pdf
   └── dev_logs.log

   You will be prompted to input the authentication token. You can leave it empty as this can be done afterward by directly editing the configuration file.

3. Configure the project (add or edit TOKEN and ZENODO_URL)

   cat lycophron.cfg
   
   # TOKEN = 'CHANGE_ME'
   # ZENODO_URL = 'https://zenodo.org/api'

4. Create a CSV file from a template

   Generate a template with all the fields:

   lycophron gen-template --filename output.csv --all

   Add custom fields (e.g. dwc and ac) to the template:

   lycophron gen-template --filename output.csv --custom "dwc,ac"

5. Fill in the metadata and load the file

   lycophron load --inputfile output.csv

6. Publish to Zenodo

   lycophron publish

Getting started

• Installation
• Commands
• Configuration
• Metadata
• How to create a CSV
• Development
• Known issues

Installation

Requirements

• Python v3.11+

Install from GitHub using pip or pipx:

pipx install "lycophron @ git+https://github.com/plazi/lycophron.git@v1.0.0

[!NOTE] In the future Lycophron will be published on PyPI and just require pip install lycophron.

Linux/macOS

It is recommended to use a virtual environment to install and run the application.

To create a virtual environment named lycophron, run the following command in your terminal:

python3 -m venv lycophron
source lycophron/bin/activate

To install the CLI tool for development, clone this repository and run:

# For local development
uv pip sync requirements-dev.txt
uv pip install -e .[dev]

Commands

init

This command initializes the project, creates necessary configuration files, and sets up a local database.

init [name] : initialize the app with the given name init --token : initialize the app with a token

:warning: Adjust configurations in the generated files to meet specific upload requirements.

validate

This command validates the current project. I.e. it checks whether the application is properly configured, the directory structure is correct and the metadata is valid and ready to be loaded locally.

validate --filename : validate the project and the given file.

gen-template

This command generates a CSV file from the default template, containing all the required headers. The template can either be generated with all the fields or by explicitely adding custom fields on top of the required ones.

gen-template --filename : creates the output file in the given path and name. gen-template --all : creates a template using all fields (required and custom fields). gen-template --custom "x,y,z" : creates a template using the required fields plus the given custom fields.

load

This command loads records from a local file into the local Database, ensuring they are ready to upload to Zenodo.

load --inputfile: load records from a given file to a local DB

publish

Publishes the previously loaded records to Zenodo.

This command specifically targets records that are currently unpublished. Importantly, this operation is designed to be executed multiple times, allowing for a phased or incremental approach to publishing records as needed.

publish: publish records to Zenodo

Configuration

Supported metadata

How to create a .csv

• Generate the template by running lycophron gen-template and fill in the metadata.
• File names must match the files under the directory /files/

[!TIP] We also provide a sample generated sheet import template with some pre-filled values.

[!WARNING] When working with fields defined as a list in the CSV file, it is essential to separate each item with a new line ("\n"). This ensures proper formatting and accurate representation of the list structure in the CSV file, thus allowing Lycophron to parse the values correctly.

Example:

Known issues

• DOIs are generated on demand, existing or external DOIs are not accepted. issue
• Metadata is compliant with legacy Zenodo, not RDM
• Output is not clear for the user. E.g. user does not know what to run next, how to fix issues with the data, etc.
• init does not create the needed file structure, still requires manual intervention (e.g. creation of ./files/)
• Linking between rows is not supported. E.g. row 1 references a record from row 2

Development

Dependency management

To manage Python dependencies, Lycophron uses uv.

Generate dependency files

You should not need to do this. More often, you will want to just bump all dependencies to their latest versions. See upgrading dependencies

# Generate requirements.txt
uv pip compile pyproject.toml > requirements.txt

# Generate requirements-dev.txt
uv pip compile pyproject.toml --extra tests > requirements-dev.txt

Add a new dependency

To add a new dependency, add it in the dependencies section of the pyproject.toml file. Add testing/development dependencies under [project.optional-dependencies] section's tests extra.

# Update the requirements.txt file
uv pip compile pyproject.toml --upgrade-package <new-package>

# Update the requirements-dev.txt file
uv pip compile pyproject.toml --extra tests --upgrade-package <new-package>

Upgrade all dependencies

To upgrade all dependencies in development, run the following command in your terminal:

uv pip compile pyproject.toml --extra tests --upgrade

and for production:

uv pip compile pyproject.toml --upgrade