Cross packing?

[Nadav-Ruskin]

Does dh-virtualenv support cross packing at all? I find myself stuck in a loop here, using an amd64 Debian Stretch docker container:

root@4f340505738a:/new/work# apt-get install dh-virtualenv
 . . . cut output for brevity . . .
Setting up dh-virtualenv (1.0-1) ...
W: APT had planned for dpkg to do more than it reported back (32 vs 38).
   Affected packages: python:amd64
root@4f340505738a:/new/work# dpkg-buildpackage -us -uc -b --host-arch=armhf
dpkg-buildpackage: info: source package configsite
dpkg-buildpackage: info: source version 0.1-1
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Nadav Ruskin <ruskin.nadav@gmail.com>
dpkg-architecture: warning: specified GNU system type arm-linux-gnueabihf does not match CC system type x86_64-linux-gnu, try setting a correct CC environment variable
dpkg-buildpackage: info: host architecture armhf
 dpkg-source --before-build work
dpkg-checkbuilddeps: error: Unmet build dependencies: python
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
root@4f340505738a:/new/work# apt-get install python:armhf
 . . . cut output for brevity . . .
Removing dh-virtualenv (1.0-1) ...
Removing python (2.7.13-2) ...
 . . . cut output for brevity . . .
Setting up python:armhf (2.7.13-2) ...
root@4f340505738a:/new/work# dpkg-buildpackage -us -uc -b --host-arch=armhf
dpkg-buildpackage: info: source package configsite
dpkg-buildpackage: info: source version 0.1-1
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Nadav Ruskin <ruskin.nadav@gmail.com>
dpkg-architecture: warning: specified GNU system type arm-linux-gnueabihf does not match CC system type x86_64-linux-gnu, try setting a correct CC environment variable
dpkg-buildpackage: info: host architecture armhf
 dpkg-source --before-build work
dpkg-checkbuilddeps: error: Unmet build dependencies: dh-virtualenv (>= 0.8)
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
root@4f340505738a:/new/work#

python:armhf overrides dh-virtualenv and vice versa! What should I do? Should I try qemu?

[jhermann]

Use a PI3 or Scaleway as a build machine might be less frustrating? ;)

[Nadav-Ruskin]

Is this a dead no? That's sad. I can cross compile and pack C++ but not Python? That just sounds wrong!

My company has access to AWS, and I've already configured it to play nice with Jenkins and whatnot. I'm trying to add as few elements to this development-deployment system as possible.

[Nadav-Ruskin]

I'd like to report success using qemu. I've installed the qemu-user-static and binfmt-support packages, and use the following dockerfile:

FROM arm32v7/debian:latest

RUN apt-get update ; apt-get -y upgrade ; apt-get update
RUN apt-get -y install python3 sudo debhelper dh-virtualenv dpkg-dev python3-venv

It only seems to work sometimes however. I read that qemu is buggy so maybe I'm not doing anything wrong, it seems to fail every few tries...

The resulting package works 100% fine, however.

I'd like to keep this issue open until I get a definitive answer from dh-vertualenv's owner.

[jhermann]

@nailor the above seems like a worthwhile addition to the docs.

[jhermann]

@Nadav-Ruskin without a log of the 'sometimes' failures it is hard to say why. network issues (to the repos) can play a role, too (which is why increasing apt timeouts in dockerfiles is not a shabby idea).

[Nadav-Ruskin]

I didn't expand on it because I haven't researched it fully yet. I'll report further conclusions later. I might even find a different solution.

[nailor]

Adding this to the docs sounds really neat idea! Anyone up for crafting a PR for some Tips&Tricks on cross-arch builds?

I'm also in the boat of intermittent failures being due to qemu/docker/something else. Probably can be sorted out with more digging, but at least I've had all kinds of fun with qemu back in the days..

[jhermann]

On the topic of tips & tricks, do we have "official" docs on excluding manylinux1 stuff and so on? That I could do…

⇒ https://github.com/1and1/debianized-sentry/blob/0f17d2239d6dee5fde6ccbf0497aefba6461fd78/debian/rules#L19-L23

Also, I'll likely add "build package in container" to the above project soon, so I can build Stretch on my Ubuntu workstation.

[jhermann]

Speaking of docs, want

https://speakerdeck.com/jhermann/devops-karlsruhe-meetup-2018-02-20

added as a link to the README?

[nailor]

Nice presentation! Yeah, there could be some sort of 'In the wild' section. I got a talk or two to add there too 👍

[nailor]

And while it, no, we have not touched the manylinux1 stuff at all and I bet people have similar battles as you do

[jhermann]

See above issue link. I'll leave the README to you.

[Nadav-Ruskin]

Haven't looked further into qemu problems yet, but I did make a public example version of my project: https://github.com/Nadav-Ruskin/configsite

I feel like it has value to people wanting to use dh-virtualenv, please consider adding a link to it in the readme or wiki.

[RafalSkolasinski]

Is this working? I tried the minimal Dockerfile posted above and I got cryptic error:

Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM arm32v7/debian:latest
 ---> a0f4bfa9bc90
Step 2/3 : RUN apt-get update ; apt-get -y upgrade ; apt-get update
 ---> Running in 611a17fa8ff7
standard_init_linux.go:211: exec user process caused "no such file or directory"
The command '/bin/sh -c apt-get update ; apt-get -y upgrade ; apt-get update' returned a non-zero code: 1

[evilaliv3]

Have anybody dealt with this and has a recipe to do cross compile with dh-virtualenv at least for the following architectures i386, amd64 and arm? thank you!

[hanrok]

hi, for those who ended up here and frustrated. I managed to create a Debian package using this Dockerfile on a Windows machine:

FROM arm64v8/ubuntu:18.04

RUN apt update
RUN apt install -y \
                build-essential \
                debhelper \
                devscripts \
                equivs \
                python \
                python3.6 \
                python3-venv \
                python3.6-dev \
                dh-virtualenv
...