Using Litestream with no Procfile (such as a dockerized Kamal deploy)

[5280bm]

If not using a Procfile, need a way to get the Litestream initalize directive into the startup.

On my Kamal deploy, I am currently running my Litestream initializer in my Docker Entry file (bin/docker-entrypoint) like such:

#docker-entrypoint

#!/bin/bash -e

# If running the rails server then create or migrate existing database
if [ "${1}" == "./bin/rails" ] && [ "${2}" == "server" ]; then
  ./bin/rails db:prepare
  litestream replicate -config /rails/config/litestream.yml &
fi

exec "${@}"

Note, I am installing Litestream and running it with a command in my Dockerfile

# Litestream
ADD https://github.com/benbjohnson/litestream/releases/download/v0.3.13/litestream-v0.3.13-linux-amd64.tar.gz /tmp/litestream.tar.gz
RUN tar -C /usr/local/bin -xzf /tmp/litestream.tar.gz

[fractaledmind]

We need to figure out the best way to integrate this usage pattern into the gem... 🤔

[fractaledmind]

@5280bm How do you run additional processes, like Sidekiq or SolidQueue in your setup?

[5280bm]

@5280bm How do you run additional processes, like Sidekiq or SolidQueue in your setup?

Using the Litestack gem to handle everything - of course with configuration - for litecache, litejobs, etc. Running it as true monolith.

[clayton]

I'm also running Litestack and deploying with Kamal. Would like to use this gem, but will stick with running it as a service on the container for now as described in @fractaledmind's blog post Enhancing your Rails app with SQLite: Setting up Litestream

[fractaledmind]

@clayton Could you detail your setup? I'd love to make the gem useful in a Docker context, but I simply don't use Docker and Kamal for my apps, so I don't know enough.

I presume you download and compile the Litestream executable from a Docker image in your Dockerfile. How do you install it as a systemd service? Is that also done in the Dockerfile? How do you start the replication process via the service? Do you do any monitoring of that process/service? Have you ever restored a backup? How exactly did that process look like?

The more I can understand what it looks like to use Litestream with Docker, Kamal, and systemd the better I can accommodate the gem.

[clayton]

I'm using the bash setup script from your blog post:

#!/usr/bin/env bash
set -e

# Load environment
source /home/deploy/.bashrc

# Determine architecture of current env
arch=$(dpkg --print-architecture)
# Manually set the Litestream version number we are using
version="v0.3.13"

# Download the latest .deb file
wget "https://github.com/benbjohnson/litestream/releases/download/$version/litestream-$version-linux-$arch.deb"

# Install that .deb file using dpkg
dpkg -i "litestream-$version-linux-$arch.deb"

# Verify it is installed
echo "Litestream version:"
litestream version

# Enable Litestream to run continuously as a background service
systemctl enable litestream

# Start Litestream running continuously as a background service
systemctl start litestream

# Verify the service is running
echo "Litestream service logs:"
journalctl -u litestream

In my Litestream config, I am directly referencing the production sqlite file that's on the host filesystem.

It's all very manual and outside of docker. I'm installing and running Litestream on the host VPS, configuring Litestream to reference the files to be synced on the host VPS, and manually adding ENV secrets for communication with the backup service.

So, not very automated or ideal given Kamal/Docker paradigms, but gets the job done.

[fractaledmind]

I'm closing after the Puma plugin work as released