search

sile-typesetter / casile

The CaSILE toolkit, a book publishing workflow employing SILE and other wizardry.
GNU Affero General Public License v3.0
54 stars 6 forks source link
books casile ebook-generator epub-generation kindlegen pandoc pdf pdf-generation publishing sile typesetting typesetting-system

readme

CaSILE toolkit

Rust Test Status Docker Build Status Rust Lint Status Lua Lint Status Reviewdog Lint Status
Chat on Gitter Conventional Commits Commitizen Friendly

The CaSILE toolkit is a build system that glues together a large collection of tools into a cohesive system that automates book publishing from start to finish. The concept is to take very simple, easily edited content and configuration files as input and turn them into all the artifacts of a finished product with as little manual intervention as possible. Plain text document formats and a small amount of meta data are transformed automatically into press ready PDFs, E-Books, and rendered promotional materials.

In traditional publishing workflows the closer a book is to production the harder becomes to work with. The pipeline ‘narrows’ to more and more advanced (complex/expensive) software and more and more restricted access. CaSILE completely eschews this limitation completely automating all the ‘later’ production stages. By automating the production workflow from end to end all the normal sequence restrictions are removed. Book exterior design can be done at any stage of the process. Book interior design can be done at any stage of the process. Copy editing can happen simultaneously by different editors on different parts of a book. Because the pipeline doesn’t narrow as projects progress and the content is always front and center the only restrictions on the workflow are those dictated by you for the project, not by the tooling used.

CaSILE (pronounced /ˈkɑːs(ə)l/ just like ‘castle’) started out life as a submodule called avadanlik included inside my book project repositories (avadanlık being a Turkish word for something like a tackle box). As most of the parts revolved around SILE, in my head at least CaSILE became Caleb’in Avadanlığı birlikte SILE, roughly translating to “SILE with Caleb’s Toolkit”. Initially everything was hard coded in Turkish, but eventually I added localization for English and generalized most of the tooling so it can be used for books in nearly any language.

Status

I’ve published dozens of books and other projects this way and have more in progress. It’s now used by at least 3 publishing companies. In other words it Works for Me™ but your millage may vary. This tool started out as just some scripts baked into one book project. Then I worked on another book and copied the scripts over to get started. When I hit book number 3, it dawned on me I should make my tools more modular and just include them in each project. About this time I knew I wanted to open source it if it proved useful for more than one type of book. That day came and went. One day I just decided to throw it out there so that it would be easier to explain what I was doing. As such in many ways it is hard coded to my publishing needs any adaption to be more flexible only happens as people request or contribute the changes.

There are several different ways to use CaSILE, with or without installation. Originally (through v0.2.x) CaSILE focused on use as a submodule to Git projects. Beginning with v0.3.0 the primary focus has been on use as CLI tool completely separate from any project.

Setup

CaSILE can be installed and run locally as a standard CLI program if you’re willing to setup the extensive list of dependencies.

As an easier alternative to installing all the dependencies yourself, everything may be run prepackaged together as a single Docker container.

In addition to being run locally, CaSILE can also be run from nearly any remote CI platform. If your book project is on GitHub, you can add CaSILE to any workflow as a GitHub Action. If your book project is hosted on GitLab, you can easily configure it to run in GitLab CI.

Of course it is also possible to mix and match.

Local Native Setup

If you happen to be using Arch Linux the casile package on the AUR is all you need. Also a casile-git recipe is available, and packages (including all dependencies) can be installed directly from this repo) for easy setup. For any other platform you’ll either need to be up for an adventure, see building from source (or just use Docker). It is possible to run on macOS if you spend some time pulling in dependencies from Homebrew and elsewhere. Windows support will almost certainly require considerable monkey business; not my circus, not my monkeys.

Local Docker Setup

Use of a Docker container can make it a lot easier to get up and running because you won’t need to have a huge collection of dependencies installed. Ready made containers are available from either Docker Hub or GitHub Container Registry. Download (or update) an image using docker pull docker.io/siletypesetter/casile:latest or docker pull ghcr.io/sile-typesetter/casile:latest. Note latest will be the most recent stable tagged release, or you may substitute a specific tag (e.g. vX.Y.Z), master for the more recent Git commit build, or v0 for the more recent tagged release in that major series.

Optionally you may build a docker image yourself. From any CasILE source directory (a Git clone extracted source package), configure using ./configure --disable-dependency-checks, then build using make docker. The resulting image will be available on your system as sile-typesetter/casile:HEAD.

In order to invoke CasILE from Docker you need to pass in your project files on a volume that will also serve as a place it can write it’s output. The full Docker run command can be substituted anywhere you would invoke CaSILE. For convenience you’ll probably want to give yourself an alias:

alias casile='docker run -it --volume "$(pwd):/data" ghcr.io/sile-typesetter/casile:latest

Save this in your shell’s rc file (e.g. ~/.bashrc) to persist the alias. This substitution should work anywhere and with any arguments you would have run casile for.

GitHub Action Setup

Use as an Action follows the traditional GitHub Action configuration pattern. You can specify the exact version you want, v0 for the most recent tagged release in the same major version sequence, latest far the very latest tagged release of any sequence, or master for the latest development build.

- name: CaSILE
  uses: sile-typesetter/casile@latest

If no arguments are passed, by the Action will default to running casile make -- default. You can pass your own arguments using the args input parameter. The DISTDIR value is output automatically and can be used to post artifacts from your build. A complete workflow example .github/workflows/casile.yml with customized targets and artifact posting might look like this:

name: CaSILE
on: [push, pull_request]
jobs:
  casile:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - id: casile
        uses: sile-typesetter/casile@latest
        with:
          args: make -- pdfs epub renderings
      - name: Upload artifacts
        uses: actions/upload-artifact@v2
        with:
          name: ${{ steps.casile.outputs.DISTDIR }}
          path: ${{ steps.casile.outputs.DISTDIR }}

Another useful paradigm is to run your steps inside the container:

name: CaSILE
on: [push, pull_request]
jobs:
  casile:
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/sile-typesetter/casile:latest
      options: --entrypoint=bash
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: make
        run: |
          casile make -- pdfs epub renderings
      - name: Upload artifacts
        uses: actions/upload-artifact@v2
        with:
          name: pub
          path: pub

If you are just starting from scratch, consider using the casile-template repository to initialize your project. For more ideas and complex examples, check out the casile-demos repository.

GitLab CI Setup

Setup your job to use the CaSILE image and version of your choice, but disable the default entry point:

image:
  name: "siletypesetter/casile:latest"
  entrypoint: [""]
script:
  - casile make

Unfortunately GitLab CI can’t name artifacts dynamically (see upstream report), so you’ll need to define the DISTDIR variable yourself. A complete pipeline example .gitlab-ci.yaml with customized targets and artifact posting might look like this:

default:
  image:
    name: "siletypesetter/casile:latest"
    entrypoint: [""]
variables:
  DISTDIR: $CI_PROJECT_NAME-$CI_JOB_NAME_SLUG-$CI_COMMIT_SHORT_SHA
casile:
  script:
    - casile make -- pdfs epub renderings
  artifacts:
    name: $DISTDIR
    paths: [ ./$DISTDIR/* ]

Dependencies

Note if you use any of the distro packages, Docker containers, or CI configurations listed in Setup you don't need to worry about these dependencies.

CaSILE glues together a lot of different open source tools to assemble a complete publishing tool chain. Behind the scenes this is very messy business. In order to make everything work I’ve had to use an eclectic variety of software. All of these are open source and available across platforms, but I only personally test on Linux.

All of the following are utilized in one way or another. Currently the toolkit assumes all the following are present, but as not all of them are used to build all resources it could be possible to make this more selective. For example not having the ray tracing engine would just mean no fancy 3D previews of book covers, but you could still build PDFs and other digital formats. Not having Node would mean no Bible verse format normalization, but you should still be able to build books. Not having ImageMagick would mean no cover images, but you could still process the interior of books. On the other hand not having GNU Make, Pandoc, or SILE would of course be fatal.

In addition to run-time dependencies, compiling the CLI interface (optional) requires a Rust build toolchain. Once built the CLI requires no dependencies to run.

Until other distros have packages, perhaps the most definitive list of dependencies is the Arch Linux package meta-dataa. You will need to translate the package names for your platform but everything is listed there.

Companion tools

You’ll probably want some other things not provided by CaSILE as well. CaSILE takes care of transforming sources to finished outputs, but leaves you to edit the sources and view the outputs yourself. For starters a text editor for working with Markdown & YAML sources will be a must-have. Options abound here and are mostly out of scope, but think Marktext, Zettlr, Atom, VSCode, Sublime, Vim, etc. CaSILE also assumes your book project is tracked in Git, so a client such as the CLI tools or a GUI like GitAhead, Fork, Sourcetree, GitKraken, Tower, or a plugin specific to your editor of choice is a must-have. Of course you’ll want a way to view generated PDFs. I recommend one that auto updates on file changes; I use zathura), but Okular and quite a few others also support this. An image viewer and an E-Book reader like Calibre are also useful.

Building from Source

  1. Clone the Git repository or download and extract a source archive or source release package.

  2. Change to the directory, configure for your system, and build the tools:

    # Only for git clones or source archives...
$ ./bootstrap.sh

# Configure & build
$ ./configure
$ make
    $ make install

    Note if you don't plan to install to your system but would like to compile and run from the source directory (e.g. for development work on CaSILE itself), use ./configure --datarootdir=$(cd ..;pwd); make -B, then add the CasILE source directory to your $PATH.

Input

CaSILE makes a number of assumptions about the content of your project repository, but how exactly you organize your git repos is still flexible. For example I have some single books in their own repositories, some series of books where the repository holds the whole series, others with different books with the same publisher/copyright status lumped together (and worked on in branches), a set of tracts by assorted authors but published together in another repository, etc. CaSILE assumes there is some relation between sources in each repository so granular repositores give more complete control, but each resource in a single repository can also be customized. You’ll have to consider your own workflow and how projects share resources. Note that common resources, say defaults for a publisher, can be shared in another submodule(s).

A book project would minimally consist of at least the following:

There might be more related assets of course, for example a cover background image:

Optionally books may be split into chapters:

Output

In return, CaSILE will output

Usage

Build whatever resources you need. Assuming you have a book source file my_book.md and accompanying meta data at my_book.yml, the general syntax for generating resources would be:

$ casile make -- my_book-<layout>-<options>.<format>

For example to build the press ready PDF for an Octavo size version:

$ casile make -- my_book-octavo-hardback-cover.pdf

Or to build a 3D rendering of the front cover at Halfletter size:

$ casile make -- my_book-halfletter-paperback-3d-front.jpg

See also the CaSILE demos repository for a sample book project layout.

Settings

CaSILE has a lot of knobs to fiddle with, and almost anything can be changed. The main trick is understanding where to make changes, because order matters.

As a broad overview from least specific to most:

Project parameters

Most settings apply to a whole project (repository). To override the defaults set them in your project’s casile.mk

Build time settings

These settings are usually not changed except at run time. You may set them in your rules file such as casile.mk but they would typically be set as environment variables or on the command line to get other-than-default behavior for a specific build.

Hooks

These are functions that can be defined in your project’s Makefile to add additionally functionality at various points in the process. You make use either single or multiline syntax as desired, but note the input, output, and variables passed will be the same either way. On the other hand each hook has its own usage so note the context it runs in.