PackPack is a simple tool to build RPM and Debian packages from git repositories:
Fast reproducible builds using Docker containers
Semantic versioning based on annotated git tags
Support for all major Linux distributions as targets
PackPack works best with GitHub, Travis CI and PackageCloud:
Push your code to GitHub
Build packages using Travis CI
Host repositories on PackageCloud
Watch a demonstration of PackPack.
PackPack is designed by folks from Mail.Ru Group, a leading Internet company in Europe, to automate release management cycle of open source products as well as of proprietary software.
Tarantool, an open-source general-purpose database and an application server, has dozens of git commits per day and this number is constantly increasing month after month. In order to deliver the best user experience and offer enterprise-level support quality, the Tarantool team packages almost every git commit from four git branches for (!) fifteen various Linux distribution.
Traditional tools, like mock
and pbuilder
, were tremendously slow and
ridiculously overcomplicated. Customers had to wait hours for hotfix
packages and the project paid thousands of dollars annually for hardware and
electricity bills. Such costs are unacceptable for most "free as in speech"
open-source projects.
PackPack has reduced push-to-package time from hours to minutes. Tarantool team were even able to package all complementary modules, addons and connectors using this tool. Tarantool users now can also package their own proprietary modules in the same manner as official packages.
Of course, PackPack itself is packaged using PackPack.
Distributions:
Archictectures:
i386
x86_64
armhf
(32-bit ARM with hardware floating-point)aarch64
(64-bit ARM)The actual list of distribution is available on Docker Hub. Please file an issue if you want more.
Install git
, docker
and any posix-compatible shell
(bash, dash, zsh, etc.).
The complicated one is Docker, please see the detailed guide on
docs.docker.com web-site.
Add RPM spec
to rpm/
folder of your git repository. The best way to
create a new spec
file for your product is to find an existing one for
a similar software, rename and then modify it. See Fedora Git and
Fedora Packaging Guidelines for details. Some examples are available
from tarantool/modulekit repository.
Add debian/
folder to your git repository, as usual. Debian has
complicated package structure and we strongly recommend to find a similar
package in the official repositories,
download it using apt-get source package
command, copy and paste and
then modify debian/
directory. Some examples are available from
tarantool/modulekit repository.
Create an annotated major.minor
git tag in your repository.
PackPack will automatically set patch
level based on the commit number
from this tag in order to provide major.minor.patch
semantic versioning:
$ git tag -a 1.0
$ git describe --always --long
1.0-0-g5c26e8b # major.minor-patch = 1.0-0
$ git push origin 1.0:1.0 # Push to GitHub
myproject$ git clone https://github.com/packpack/packpack.git packpack
myproject$ OS=fedora DIST=24 ./packpack/packpack
build/
directory:myproject$ ls -1s build/
total 112
76 myproject-1.0.2-0.fc24.src.rpm
36 myproject-devel-1.0.2-0.fc24.x86_64.rpm
Of course, PackPack can also be installed from DEB/RPM packages:
# For Debian, Ubuntu and other Debian-based
curl -s https://packagecloud.io/install/repositories/packpack/packpack/script.deb.sh | sudo bash
# For Fedora, RedHat, CentOS and other RPM-based
curl -s https://packagecloud.io/install/repositories/packpack/packpack/script.rpm.sh | sudo bash
See PackPack Repositories for additional instructions.
PackPack performs the following steps:
Checks if it runs in CI environment reading predefined variables from CI and sets its appropriate name in 'CI' variable from the list: 'appveyor', 'circle', 'github', 'gitlab', 'travis'. If CI is not detected 'CI' variable will be empty. 'CI' is passed to RPM spec as '_ci' macro and as 'CI' environment variable to DEB rules.
A Docker container is started using packpack/packpack:$OS$DIST
image.
The source repository is mounted to the container as a read-only volume.
major.minor.patch
version is extracted from git describe
output.
A source tarball (product-major.minor.patch.tar.gz
) is packed from
files added to git repository.
For RPM package:
spec
file is copied from rpm/
, Version:
tag is updated
according to extracted major.minor.patch
version, %prep
is updated to match the source tarball file name.product-major.minor.patch-release.dist.src.rpm
) is
built from the source tarball using generated spec file.dnf builddep
or yum-builddep
.
Docker images already have a lot of packages pre-installed to speed up
the build.For Debian packages:
debian/changelog
is bumped with extracted major.minor.patch
git version.mk-build-deps
tool.
Docker images already have a lot of packages pre-installed to
speed up the build.Resulted packages, tarballs and log files are moved to /build
volume,
which is mounted by default to ./build
directory of your git repository.
PackPack is designed to use with GitHub, Travis CI and PackageCloud.
Register free PackageCloud account and create a repository.
Add your GitHub project to Travis CI.
Add the following environment variables to the project settings on Travis CI:
PACKAGECLOUD_TOKEN=<token>
(secret)PACKAGECLOUD_USER=<username>
(public)PACKAGECLOUD_REPO=<reponame>
(public)Enable PackPack magic in .travis.yml
file:
sudo: required
services:
- docker
cache:
directories:
- $HOME/.cache
language: C
env:
matrix:
- OS=el DIST=6
- OS=el DIST=7
- OS=fedora DIST=24
- OS=fedora DIST=25
- OS=ubuntu DIST=trusty
- OS=ubuntu DIST=precise
- OS=ubuntu DIST=xenial
- OS=ubuntu DIST=yakkety
- OS=debian DIST=jessie
- OS=debian DIST=wheezy
- OS=debian DIST=stretch
- OS=ubuntu DIST=xenial ARCH=i386
- OS=debian DIST=jessie ARCH=i386
script:
- git submodule update --init --recursive
- git describe --long
- git clone https://github.com/packpack/packpack.git packpack
- packpack/packpack
deploy:
# Deploy packages to PackageCloud
provider: packagecloud
username: ${PACKAGECLOUD_USER}
repository: ${PACKAGECLOUD_REPO}
token: ${PACKAGECLOUD_TOKEN}
dist: ${OS}/${DIST}
package_glob: build/*.{deb,rpm}
skip_cleanup: true
on:
branch: master
condition: -n "${OS}" && -n "${DIST}" && -n "${PACKAGECLOUD_TOKEN}"
Push changes to GitHub repository to trigger Travis CI build.
Check Travis CI logs and fix packaging problems, if any. Click to see how.
Get packages on your PackageCloud repository. Click to see how.
???
Star this project on GitHub if you like this idea.
PROFIT
That's it.
BTW, Travis CI allow to exclude some builds from matrix, see an example in Tarantool GitHub repo.
PackPack can be configured via environment variables:
OS
- target operating system name, e.g. fedora
or ubuntu
DIST
- target distribution name, e.g 24
or xenial
ARCH
- target architecture, like on Docker Hub:
i386
x86_64
armhf
aarch64
It is possible to use ARCH=i386
on x86_64
host and
ARCH=armhf
on aarch64
host, but there is no way to run ARM images on
Intel and vice versa. Docker is qemu and can't emulate foreign
instruction set.BUILDDIR
- a directory used to store intermediate files and resulted
packages (default is ./build
).PRODUCT
- the name of software product, used for source tarball and
source package, e.g. tarantool
VERSION
- semantic version of the software, e.g. 2.4.35
(default is extracted for git describe
).RELEASE
- the number of times this version of the software has been
packaged (default is 1).ABBREV
- abbreviation of build metadata (default is git hash, extracted
from git describe
).TARBALL_COMPRESSOR
- a compression algorithm to use, e.g. gz, bz2, xz
(default is xz).CHANGELOG_NAME
, CHANGELOG_EMAIL
, CHANGELOG_TEXT
- information
used to bump version in changelog files.DOCKER_REPO
- a Docker repository to use (default is packpack/packpack
).USE_LOCAL_IMAGE
- if not empty try to use local docker image first (by default docker image is pulled from the DOCKER_REPO
).CCACHE*
- Config variables for ccache, such as CCACHE_DISABLEPRESERVE_ENVVARS
- a comma separated list of environment variables to
preserve.ABUILD_KEY
- a private key for signing alpine packages generated by
abuild-keygen
. Example show how to create alpine keys.
docker run -it --rm alpine:3.16 sh -c 'apk add alpine-sdk; abuild-keygen -n; cat ~/.abuild/*'
See the full list of available options and detailed configuration guide in pack/config.mk configuration file.
The actual list of distribution is available on [Docker Hub] (https://hub.docker.com/r/packpack/packpack/tags/).
PackPack is written on Makefiles and contains less than 300 lines of code. We've tried different variants, like Python, but GNU Make is actually the simplest (and fastest) one.
Any pull requests are welcome.
Please feel free to fork this repository for experiments. You may need to create your own repository on Docker Hub. Click to see how.
Watch a demonstration of PackPack. Please feel free to contact us if you need some help:
PackPack can be installed as a regular system tool from RPM/DEB packages.
Check out PackPack Repositories on PackageCloud.
Please "Star" this project on GitHub to help it to survive! Thanks!