Add convenient packaging infrastructure

[amotl]

Introduction

By making a start to #3 in order to bring this program to a wider audience, this adds a lightweight but convenient packaging infrastructure.

Goals

• We should provide ready-to-go packages for popular *nix operating systems on the GitHub release page. However, this should be as effortless as possible. Nobody has time to write and maintain individual packagers supporting the plethora of Linux/Unix variants out there.
• Providing armhf packages alongside amd64 will unlock support for all kinds of Single-board computers like BeagleBone, RaspberryPi and friends.

History

The package building subsystem was originally conceived for Kotori the other day. ocli now has the pleasure of profiting from a trimmed and generalized version of it.

Outlook

During cherry-picking the gems over from the Kotori repository, I tried to make most spots as generic as possible, so that it can be extended to other operating systems and architectures. Also, @jpmens and @ckrey might want to reuse it 1:1 for qtripp and friends as it works almost plug & play now.

[amotl]

About

The packaging process is completely automated based on Docker and fpm, which can build packages for the following targets:

• deb
• rpm
• solaris
• freebsd
• tar
• directories
• Mac OS X .pkg files (osxpkg)
• pacman (ArchLinux) packages

Details

• Currently, the infrastructure has been prepared to build Debian packages for both amd64 and armhf architectures through corresponding make targets.
• Whether you would like to build against Debian 9.x stretch or Debian 10.x buster is completely up to you. This can be controlled by providing respective arguments on the command line, see "Usage" section.

Usage

Read more about its usage within Packaging README.

Demo

https://github.com/daq-tools/owntracks-publisher/releases

Backlog

There are some more things to polish. We are tracking them at #5.

[jpmens]

@amotl thank you for this contribution.

The changes to the Makefile mean that it no longer builds on non-gmake systems, so I won't take it as it stands; the packaging bits will have to be detached from the Makefile, probably best into their own independent packaging/Makefile?

[amotl]

Dear @jpmens,

I changed the packaging to work from within the packages directory as you suggested. There, it has a separate Makefile for doing its work.

With kind regards, Andreas.

[jpmens]

Merged, thank you

[amotl]

@jpmens

The packaging process is completely automated based on Docker and fpm.

I see that you have been using fpm already.

• https://github.com/owntracks/recorder/blob/0.8.4/etc/centos/fpm-make.sh
• https://github.com/owntracks/recorder/blob/0.8.4/etc/debian/fpm-make.sh

So, I recognize this is actually nothing new to your toolbox. However, the Docker integration now makes it possible to build that jazz without even having the respective Linux distribution installed on a real machine.

[jpmens]

@amotl I didn't know that, thanks.

[amotl]

The Docker integration with fpm now makes it possible to build that jazz without even having the respective Linux distribution installed on a real machine.

I didn't know that, thanks.

Just try to invoke these commands [1] on your workstation. They should work out of the box. If not, please let me know.

[1] https://github.com/owntracks/ocli/blob/master/packaging/README.rst

[jpmens]

Be warned: I have no idea what's going on:

$ docker login
Username: jpmens
Password: xx
Login Succeeded

$ make  debian-package arch=amd64 dist=buster version=0.7.0
/Applications/Xcode.app/Contents/Developer/usr/bin/make deb-build name=owntracks-publisher
Building package owntracks-publisher version 0.7.0 for architecture amd64 and distribution buster.
docker build --tag owntracks/owntracks-publisher-build-buster-amd64:0.7.0 --build-arg BASE_IMAGE=owntracks/baseline-buster-amd64:latest --build-arg DISTRIBUTION=buster --build-arg VERSION=0.7.0 --build-arg NAME=owntracks-publisher --file dockerfiles/Dockerfile.builder ..
Sending build context to Docker daemon  5.353MB
Step 1/20 : ARG BASE_IMAGE
Step 2/20 : FROM ${BASE_IMAGE} AS build-environment
pull access denied for owntracks/baseline-buster-amd64, repository does not exist or may require 'docker login'
make[1]: *** [deb-build] Error 1
make: *** [debian-package] Error 2

[amotl]

@jpmens Please run the step for preparing the baseline images [1] on your local workstation. docker login is not required at all.

[1] https://github.com/owntracks/ocli/blob/master/packaging/README.rst#setup

[jpmens]

Wer lesen kann ist klar im Vorteil ...

[jpmens]

Who's GITHUB_TOKEN ist that? (642ff7c47b1697a79ab7f105e1d79f054d0bbeef)

[jpmens]

I assume I export mine.

[amotl]

Who's GITHUB_TOKEN ist that? (642ff7c47b1697a79ab7f105e1d79f054d0bbeef)

It's a completely random one. You might want to use your own token which has access to owntracks/ocli. For testing this task, my personal GITHUB_TOKEN worked on daq-tools/owntracks-publisher well, see [1].

[1] https://github.com/daq-tools/owntracks-publisher/releases/tag/0.7.0

[amotl]

make debian-package arch=amd64 dist=buster version=0.7.0

You might want to bump that to version=0.8.0 already.

[jpmens]

I'm not that thick ;-) Yes, done that.

[jpmens]

OMG.

[jpmens]

@amotl you realize what you've caused, don't you? I've smelled blood, and I now want that magical Makefile all over the show, and can we add CentOS? :-)

[amotl]

Building for CentOS

Will that work in general?

CentOS, difficult as dependencies (gpsd, mosquitto) currently not in Epel8. See https://github.com/owntracks/ocli/issues/5.

I actually have been able to satisfy these dependencies and make it build already.

# From: https://centos.pkgs.org/7/epel-x86_64/gpsd-3.10-5.20140524gitd6b65b.el7.x86_64.rpm.html
# TODO: Adjust for CentOS 8.
RUN wget https://download-ib01.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-12.noarch.rpm
RUN rpm -Uvh epel-release*rpm

RUN yum install -y gpsd-devel mosquitto-devel

Outcome

While building for CentOS 6 didn't invoke at all (can't remember the reason), It croaked for me later in the packaging process for both the amd64 and arm64v8 architecture targets on CentOS 7. I believe it was on the invocation of fpm. Haven't tried CentOS 8 though, but you will be able to adjust easily.

Preview

Build baseline image

# CentOS 7
make build-centos-baseline arch=amd64 dist=7 version=0.1.0
# defunct; make build-centos-baseline arch=arm32v7 dist=7 version=0.1.0
make build-centos-baseline arch=arm64v8 dist=7 version=0.1.0

Invoke packaging

# CentOS 7 amd64
make centos-package arch=amd64 dist=centos7 pkgtype=rpm version=0.7.0

---

I will add general packaging support for CentOS right away. However, it still has the problems outlined within this post.

[jpmens]

No hurry, no rush. Thank you. :)

[jpmens]

When you're done I will rename the package to

DIST_PACKAGE := owntracks-cli-publisher

to make it homogenous with the repo name.

[amotl]

It croaked for me later in the packaging process for both the amd64 and arm64v8 architecture targets on CentOS 7. I believe it was on the invocation of fpm.

Maybe it was because the parameters of the fpm-package helper [1] have not been adjusted appropriately, like seen within [2].

[1] https://github.com/owntracks/ocli/blob/0.8.0/packaging/builder/fpm-package [2] https://github.com/owntracks/recorder/blob/0.8.4/etc/centos/fpm-make.sh