oaipmh-scythe: A Scythe for harvesting OAI-PMH repositories.

Welcome to oaipmh-scythe, an updated and modernized version of the original sickle, now with additional features and ongoing maintenance.

CI
Docs
Package
Meta

oaipmh-scythe is a lightweight OAI-PMH client library written in Python. It has been designed for conveniently retrieving data from OAI interfaces the Pythonic way:

from oaipmh_scythe import Scythe

with Scythe("https://zenodo.org/oai2d") as scythe:
    records = scythe.list_records()
    next(records)
# <Record oai:zenodo.org:4574771>

Features

Easy harvesting of OAI-compliant interfaces
Support for all six OAI verbs
Convenient object representations of OAI items (records, headers, sets, ...)
Automatic de-serialization of Dublin Core-encoded metadata payloads to Python dictionaries
Option for ignoring deleted items

Requirements

Python >= 3.10

oaipmh-scythe is built with:

httpx for issuing HTTP requests
lxml for parsing XML responses

Installation

You can install oaipmh-scythe via pip from PyPI:

python -m pip install oaipmh-scythe

Documentation

The documentation is made with Material for MkDocs and is hosted by GitHub Pages.

Similar Projects

There are a couple of similar projects available on PyPI and GitHub, e.g. via the topics oai-pmh and oai-pmh-client. Among them are these implementations in Python:

Project	Description	Last commit
sickle	`oaipmh-scythe` is a fork of `sickle`
pyoai	`sickle` was inspired by `pyoai`
pyoaiharvester	oai-pmh harvester CLI
ddblabs-ometha	oai-pmh harvester with CLI and TUI
oai-harvest	uses `pyoai` internally
oai-pmh-harvester	uses `sickle` internally