PRONOM Release Tools

Tools for working with PRONOM releases.

• PRONOM Release Tools
  • Tooling
  • PRONOM Tools
    • Check existing
    • Release stats
  • PRONOM Stats
    • Environment
    • Ports
  • PRONOM Cron
    • Example cron
  • PRONOM Summary
  • Pronom Export
  • Developer install
  • pip
  • tox
    • Run tests (all)
    • Run tests-only
    • Run linting-only
  • pre-commit
  • Packaging
  • pyproject.toml
  • Local packaging
  • Publishing

Tooling

A summary of the tooling included. A total of three tools are packaged in this repository, and they can all be run with the -h flag to see what options are available. The three tools are:

pronom-tools
pronom-stats
pronom-cron
pronom-export

These can be installed from the Python Packaging Index (PyPi) here.

More information can be found about the commands below.

PRONOM Tools

Coordinate PRONOM release output to produce a summary about the dataset and provide links to latest release information, including links to the latest PRONOM signature files.

To run:

python -m src.pronom_tools.pronom_tools

Check existing

{
  "release_notes_http_date": "2023-11-23",
  "latest_puid": "fmt/1924",
  "version": "V116",
  "sig_file": "https://cdn.nationalarchives.gov.uk/documents/DROID_SignatureFile_V116.xml",
  "container_sig": "https://cdn.nationalarchives.gov.uk/documents/container-signature-20231127.xml",
  "x+puid_const": "x-fmt/455"
}

Release stats

{
  "release_notes_http_date": "2023-11-23",
  "latest_puid": "fmt/1924",
  "version": "V116",
  "sig_file": "https://cdn.nationalarchives.gov.uk/documents/DROID_SignatureFile_V116.xml",
  "container_sig": "https://cdn.nationalarchives.gov.uk/documents/container-signature-20231127.xml",
  "x_puid_const": "x-fmt/455",
  "pronom_data": [
    {
      "name": "Broadcast WAVE 0 Generic",
      "description": "complete",
      "signature": true,
      "identifier": "fmt/1"
    },
    {
      "name": "Microsoft Word for Macintosh Document 3.0",
      "description": "complete",
      "signature": true,
      "identifier": "x-fmt/1"
    }
  ]
}

PRONOM Stats

Provides a HTTP server to store stats about a PRONOM release based on the different scripts above.

To run:

python -m src.pronom_stats.pronom_stats

Environment

A pronom.emv file is needed that looks as follows:

# Config for PRONOM tools.
SERVER_AUTH=badf00d
SERVER_ADDR=http://127.0.0.1:26000

Ports

Ports that are used by this application:

pronom api: 26000
pronom summary site: 26001

PRONOM Cron

PRONOM Cron can be run as a cron task to update the pronom summary database on a regular basis. Run with -i to initialize, and from there, run it n- times a day to look for a new PRONOM release.

To run:

python -m src.pronom_cron.pronom_cron

Example cron

NB. the following example requires that cron-stats is setup and running so that the stats downloaded during the cron job can be stored.

To test for a new PRONOM release every four hours run:

crontab -e

And then add a task to run every four hours.

0 */4 * * * pronom-cron

For more cron examples see cron guru,

PRONOM Summary

Summarize a PRONOM by comparing the PRONOM dataset with its corresponding signature files. The output for a single file format looks as follows:

  {
    "name": "yEnc Encoded File",
    "description": "complete",
    "signature": true,
    "identifier": "fmt/1100"
  }

To run:

python -m src.pronom_summary.pronom_summary

Pronom Export

Export PRONOM reports with PRONOM export, supplying the latest known fmt identifier and a path to export to.

To run:

python -m src.pronom_stats.pronom_export

Developer install

pip

Setup a virtual environment venv and install the local development requirements as follows:

python3 -m venv venv
source venv/bin/activate
python -m pip install -r requirements/local.txt

tox

Run tests (all)

python -m tox

Run tests-only

python -m tox -e py3

Run linting-only

python -m tox -e linting

pre-commit

Pre-commit can be used to provide more feedback before committing code. This reduces reduces the number of commits you might want to make when working on code, it's also an alternative to running tox manually.

To set up pre-commit, providing pip install has been run above:

• pre-commit install

This repository contains a default number of pre-commit hooks, but there may be others suited to different projects. A list of other pre-commit hooks can be found here.

Packaging

The justfile contains helper functions for packaging and release.

justfile functions can be reviewed by calling just from the root of this repository:

Available recipes:
    clean               # Clean the package directory
    docs                # Generate documentation
    help                # Help
    package-check       # Check the distribution is valid
    package-deps        # Upgrade dependencies for packaging
    package-source      # Package the source code
    package-upload      # Upload package to pypi
    package-upload-test # Upload package to test.pypi
    pre-commit          # Run all pre-commit checks
    pronom-cron args    # Pronom cron
    pronom-export args  # Pronom export
    pronom-stats args   # Pronom stats
    pronom-tools args   # Pronom tools
    serve-docs          # Serve the documentation
    tar-source          # Package repository as tar for easy distribution
    upgrade             # Upgrade project dependencies

pyproject.toml

Packaging consumes the metadata in pyproject.toml which helps to describe the project on the official pypi.org repository. Have a look at the documentation and comments there to help you create a suitably descriptive metadata file.

Local packaging

To create a python wheel for testing locally, or distributing to colleagues run:

• just package-source

A tar and whl file will be stored in a dist/ directory. The whl file can be installed as follows:

• pip install <your-package>.whl

Publishing

Publishing for public use can be achieved with:

• just package-upload-test or just package-upload

just-package-upload-test will upload the package to test.pypi.org which provides a way to look at package metadata and documentation and ensure that it is correct before uploading to the official pypi.org repository using just package-upload.