Support docker installation on RaspberryPi

[krasi-georgiev]

some of the images don't support the arm architecture. I will look into how to add support for this.

Would you accept a PR for that?

The reason why I can't use the official RPI OS is that I need to run some other tools in docker containers and the official Rpi OS doesn't run raspbian so it would probably be quite hard to install docker on it.

[krasi-georgiev]

I have been playing with this a bit and since golang supports multi arch complication the only change in the docker files would be:

# Install TARGETPLATFORM parser to translate its value to GOOS, GOARCH, and GOARM
COPY --from=tonistiigi/xx:golang / /
# Bring TARGETPLATFORM to the build scope
ARG TARGETPLATFORM

and than in the CI use a recent docker version that supports buildx so that can push a multi arch images like:

docker buildx build --platform linux/amd64,linux/arm/v7,linux/arm/v6 -t repo/image --push .

[krasi-georgiev]

I just successfully build multi arch images

https://hub.docker.com/u/arribada

[krasi-georgiev]

I have just finished implementing this with Circle Ci and works quite well. Please confirm whether you would accept a PR for this. Automated multi arch builds with CircleCI

[brocaar]

Currently Docker images are being built using the Docker Hub automated builds, for now I would like to keep it this way. Unfortunately it does not yet support buildx: https://github.com/docker/hub-feedback/issues/1779.

Would you mind posting a full Dockerfile as a reference?

[krasi-georgiev]

What is the reason why you want to keep it with the docker hub setup?

A big advantage of having it automated with circle is that everyone would be able to contribute since the build config will be in the github

here is a working Dockerfile for the app server

FROM golang:1.12-alpine AS development

ENV PROJECT_PATH=/lora-app-server
ENV PATH=$PATH:$PROJECT_PATH/build
ENV CGO_ENABLED=0
ENV GO_EXTRA_BUILD_ARGS="-a -installsuffix cgo"

RUN apk add --no-cache ca-certificates make git bash protobuf alpine-sdk nodejs nodejs-npm

RUN mkdir -p $PROJECT_PATH
COPY . $PROJECT_PATH
WORKDIR $PROJECT_PATH

RUN make dev-requirements ui-requirements

# Install TARGETPLATFORM parser to translate its value to GOOS, GOARCH, and GOARM
COPY --from=tonistiigi/xx:golang / /
# Bring TARGETPLATFORM to the build scope
ARG TARGETPLATFORM

RUN make

FROM alpine:latest AS production

WORKDIR /root/
RUN apk --no-cache add ca-certificates
COPY ./lora-app-server.toml /etc/lora-app-server/lora-app-server.toml
COPY --from=development /lora-app-server/build/lora-app-server .
ENTRYPOINT ["./lora-app-server"]

and a circle ci build process would like something like this:

version: 2.1
jobs:
  docker_publish:
    machine:
      image: ubuntu-1604:201903-01
    steps:
      - checkout  
      - run:
          command: |
            // install the latest docker that supports buildx.
            sudo apt update 
            sudo apt install -y qemu qemu-user-static qemu-user binfmt-support
            git clone https://github.com/computermouth/qemu-static-conf.git
            sudo mkdir -p /lib/binfmt.d
            sudo cp qemu-static-conf/*.conf /lib/binfmt.d/
            sudo systemctl restart systemd-binfmt.service
            sudo apt purge -y docker-ce docker-ce-cli
            sudo curl -fsSL https://get.docker.com -o get-docker.sh
            sudo sh get-docker.sh
            sudo usermod -aG docker $(whoami)
            mkdir $HOME/.docker
            sh -c 'echo "{ \"experimental\": \"enabled\" }" >> $HOME/.docker/config.json'
            docker buildx create --name=multi
            docker buildx inspect --bootstrap multi
            docker buildx use multi
            docker login --username=$DOCKERHUB_USER --password=$DOCKERHUB_PASS

            docker buildx build --progress=plain --platform linux/amd64,linux/arm/v7 -t $DOCKERHUB_REPO/smart-connect$CIRCLE_TAG --push .

workflows:
  version: 2
  publish:
    jobs:
      - docker_publish:
          filters:
            branches:
              ignore: /.*/
            tags:
              only: /v\d+\.\d+\.\d+$/

the setup builds only for a new version tag, but automated build on every merge to the master branch can also be added.

[brocaar]

What is the reason why you want to keep it with the docker hub setup?

It is easy to configure (also with regexp based image tagging) and it works. Since buildx is is quite a new feature, I do hope that eventually this will get integrated into the Docker Hub automated build.

While it is great to see that with some changes, it is easy to cross compile to several different platforms, I'm hesitant to include features in the build-pipeline that are still marked as tech-preview / experimental and could break the build pipeline.

A big advantage of having it automated with circle is that everyone would be able to contribute since the build config will be in the github

Ideally I like to keep this configuration as small as possible, this makes the maintenance a lot easier for me + less error prone :)

[krasi-georgiev]

Ideally I like to keep this configuration as small as possible, this makes the maintenance a lot easier for me + less error prone :)

Yes I know exactly what you mean :smile:

but also more people maintaining has the same benefit. All projects I have worked on migrate to some custom circle or travis based build at some point.

I would probably keep a fork with this and will link the repo here for anyone willing to use these multi arch images. Problem is I would need to contantly sync with upstream for bugfixes and new features.

I would be using these with balena and unfortunately the config files need to be baked in to allow it to be deployed with balena. Anyone that needs some custom configs can just bind mount the custom ones and those will be used instead.

[krasi-georgiev]

I have forked all loraserver images set to auto build with Circle and pushed multi arch images to Docker hub

Here is one example: https://github.com/arribada/loraserver you can see the others under the same org.

Once circle updates the docker version to one that supports the buildx for multi arch images the circle config file will become very simple. Would you consider a PR than?

[krasi-georgiev]

this is now also available in GH actions, would you reconsider the decision?

https://github.com/marketplace/actions/docker-buildx

[krasi-georgiev]

ping

[krasi-georgiev]

I am willing to do the PR

[phlipse]

Any feedback on that topic? I would like to migrate my installation to docker on my Raspberry as well.

[abalmos]

As a show or interest, I could also use armv7 versions of these containers for deploying to open source machine telematics units (ISOBlue).

It seems I will need to maintain a fork of these repos for no purpose other then to use buildx over build.

[arctic-alpaca]

You don't need to maintain a fork. Github actions allow you to check out the ChirpStack repos and create multiarch containers from it. You should be able to fork my repo, add your own DockerHub name and token as secrets and activate GitHub actions. This allows you to get daily updated release containers for multiple architectures uploaded to your DockerHub account.

I can only verify those containers in a VM right now, so if you encounter any issues, let me know.

[sundeep95]

I have used the docker image for Raspberry Pi for installing chirpstack gateway bridge. I was able to run the docker image on Pi. But I don't find how to know GatewayID to add in ChirpStack Application server. Please let me know how to know the GatewayID from docker image/how to configure GatewayID.

[brocaar]

Hi @krasi-georgiev, for various other reasons, I have begun to migrate from CircleCI and Docker Automated Builds to GitHub Actions. I believe this does mean it will be fairly easy to add ARM builds.

[krasi-georgiev]

finally :)

look at this example the uses the official docker buildx action

https://github.com/arribada/i360/blob/master/.github/workflows/docker.yaml

you can replace crazy-max/ghaction-docker-meta with docker/metadata-action@v3

[brocaar]

@krasi-georgiev, that repo might not be public?

[krasi-georgiev]

ooh shoot, sorry

name: Docker

on:
  push:
    branches: master
    tags:
    - 'v*' # Push events to matching v*, i.e. v1.0, v20.15.10

jobs:
  main:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
            fetch-depth: 0 # Fetch all tags as well.
      -
        name: Docker meta
        id: docker_meta
        uses: docker/metadata-action@v3
        with:
          images: arribada/${{ github.event.repository.name }}
      -
        name: Set up QEMU
        uses: docker/setup-qemu-action@v1
      -
        name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1
      -
        name: Login to DockerHub
        uses: docker/login-action@v1 
        with:
          username: ${{ secrets.DOCKER_USER }}
          password: ${{ secrets.DOCKER_PASS }}
      -
        name: Build and push
        id: docker_build
        uses: docker/build-push-action@v2
        with:
          platforms: linux/amd64,linux/arm64/v8
          context: .
          file: ./Dockerfile
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.docker_meta.outputs.tags }}
          labels: ${{ steps.docker_meta.outputs.labels }}
      -
        name: Image digest
        run: echo ${{ steps.docker_build.outputs.digest }}

[krasi-georgiev]

any luck?

[krasi-georgiev]

I am willing to help out if you would be open to a PR for this. I have been maintaining a fork for too long now ;)

[brocaar]

Hi @krasi-georgiev, yes that would be great and would speed up the process a lot as I have quite a backlog of items I'm currently working on :)

[krasi-georgiev]

opened the first PR to add support for arm7 and arm8 https://github.com/brocaar/chirpstack-application-server/pull/618