Precise and Trusty on same buildbot?

mikepurvis commented 9 years ago

I gave this a try and wound up with some odd apt errors. Highlights:

dpkg-checkbuilddeps: Unmet build dependencies: ros-hydro-catkin ros-hydro-grizzly-msgs ros-hydro-rosserial-leonardo-cmake ros-hydro-std-msgs
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
dpkg-buildpackage: warning: this is currently a non-fatal warning with -S, but will probably become fatal in the future
 fakeroot debian/rules clean
...
Get:7 http://packages.ros.org precise/main amd64 Packages [634 kB]
Get:8 http://archive.ubuntu.com trusty/main amd64 Packages [1350 kB]
Get:9 http://archive.ubuntu.com trusty/main i386 Packages [1348 kB]
Get:10 http://packages.ros.org precise/main i386 Packages [634 kB]
Get:11 http://archive.ubuntu.com trusty/main TranslationIndex [3632 B]
Hit http://archive.ubuntu.com precise/main Translation-en
Hit http://archive.ubuntu.com precise/universe Translation-en
Get:12 http://archive.ubuntu.com trusty/main Translation-en [762 kB]
Ign http://packages.ros.org precise/main TranslationIndex
Ign http://packages.ros.org precise/main Translation-en
...
Internal error: the solver Install(python:i386 2.7.5-5ubuntu3 <python-rospkg:amd64 1.0.30-1 -> {python:amd64 2.7.5-5ubuntu3 python:i386 2.7.5-5ubuntu3}>) of a supposedly unresolved dependency is already installed in step 2
Internal error: the solver Install(python:i386 2.7.5-5ubuntu3 <python-rospkg:amd64 1.0.30-1 -> {python:amd64 2.7.5-5ubuntu3 python:i386 2.7.5-5ubuntu3}>) of a supposedly unresolved dependency is already installed in step 2
Internal error: the solver Install(python:i386 2.7.5-5ubuntu3 <python-rosdistro:amd64 0.3.6-1 -> {python:amd64 2.7.5-5ubuntu3 python:i386 2.7.5-5ubuntu3}>) of a supposedly unresolved dependency is already installed in step 2
Internal error: the solver Install(python:i386 2.7.5-5ubuntu3 <python-rosdistro:amd64 0.3.6-1 -> {python:amd64 2.7.5-5ubuntu3 python:i386 2.7.5-5ubuntu3}>) of a supposedly unresolved dependency is already installed in step 2
Internal error: the solver Install(python:i386 2.7.5-5ubuntu3 <python-catkin-pkg:amd64 0.2.6-1 -> {python:amd64 2.7.5-5ubuntu3 python:i386 2.7.5-5ubuntu3}>) of a supposedly unresolved dependency is already installed in step 2
Internal error: the solver Install(python:i386 2.7.5-5ubuntu3 <python-catkin-pkg:amd64 0.2.6-1 -> {python:amd64 2.7.5-5ubuntu3 python:i386 2.7.5-5ubuntu3}>) of a supposedly unresolved dependency is already installed in step 2
Internal error: found 2 (choice -> promotion) mappings for a single choice.
Internal error: found 2 (choice -> promotion) mappings for a single choice.
Unable to resolve dependencies!  Giving up...
...
19 packages upgraded, 303 newly installed, 0 to remove and 119 not upgraded.
Need to get 150 MB of archives. After unpacking 628 MB will be used.
Abort.
 -> Finished parsing the build-deps
Reading package lists...
Building dependency tree...
Reading state information...
You might want to run 'apt-get -f install' to correct these:
The following packages have unmet dependencies:
 fakeroot : Depends: libfakeroot (>= 1.20-3ubuntu2) but it is not going to be installed
 pbuilder-satisfydepends-dummy : Depends: debhelper (>= 7.0.50~) but it is not going to be installed
                                 Depends: ros-hydro-catkin but it is not going to be installed
                                 Depends: ros-hydro-grizzly-msgs but it is not going to be installed
                                 Depends: ros-hydro-rosserial-leonardo-cmake but it is not going to be installed
                                 Depends: ros-hydro-std-msgs but it is not going to be installed
E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).

The whole thing: https://gist.github.com/mikepurvis/0c9a1c0093ba6edd2c69

Seems like it's ending up with some kind of weird combination environment, where it's sucking in packages from both precise and trusty, despite it being a trusty cowbuilder. Is this an expected scenario? We should probably either fix this, or document that a Trusty buildbot-ros is only suitable for building Trusty+ ROS packages.

mikeferguson commented 9 years ago

@mikepurvis can you grab the cowbuilder-update step log?

mikepurvis commented 9 years ago

Sure: https://gist.github.com/mikepurvis/ae3516685d1bd48e0c00

mikeferguson commented 9 years ago

Hmm, I have no idea where the trusty is coming from -- its not in othermirror in the update or the build step -- I can only imagine this must be some internal issue in git-buildpackage?

mikepurvis commented 9 years ago

Two more dead ends—

I realised that the GBPs I was attempting to build this way were all from last fall, and generated using Bloom 0.4.7. However, having re-bloomed one of them with 0.5.12, I see no change.
I tried building the Docker container against an underlying Precise rather than Trusty. This eliminates the weird trusty elements from the buildbinary log, but the build still fails with a very similar error:

Reading package lists...
Building dependency tree...
Reading state information...
fakeroot is already the newest version.
You might want to run 'apt-get -f install' to correct these:
The following packages have unmet dependencies:
 pbuilder-satisfydepends-dummy : Depends: debhelper (>= 9.0.0) but it is not going to be installed
                                 Depends: ros-hydro-catkin but it is not going to be installed
                                 Depends: ros-hydro-grizzly-msgs but it is not going to be installed
                                 Depends: ros-hydro-rosserial-leonardo-cmake but it is not going to be installed
                                 Depends: ros-hydro-std-msgs but it is not going to be installed
E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).
I: Copying back the cached apt archive contents
I: unmounting /var/cache/pbuilder/ccache filesystem
I: unmounting dev/pts filesystem
I: unmounting proc filesystem
 -> Cleaning COW directory
  forking: rm -rf /var/cache/pbuilder/build//cow.12884 
gbp:error: git-pbuilder returned 1
gbp:error: Couldn't run 'git-pbuilder -uc -us'
program finished with exit code 1
elapsedTime=16.875008

The other piece of the puzzle this might be is using squid-deb-proxy to cache downloaded debs. That's pretty essential to keep things moving along, but it's possible that I've got it misconfigured somehow in the container.

mikepurvis commented 9 years ago

Disabling squid has no effect. I think it is some quirk with how the cowbuilder is being set up. Compare:

From: sudo cowbuilder --create --distribution trusty --architecture amd64 --debootstrapopts --arch --debootstrapopts amd64 --basepath /var/cache/pbuilder/base-trusty-amd64.cow --othermirror deb http://archive.ubuntu.com/ubuntu trusty main universe:

Ign http://archive.ubuntu.com trusty InRelease
Hit http://archive.ubuntu.com trusty Release.gpg
Hit http://archive.ubuntu.com trusty Release
Hit http://archive.ubuntu.com trusty/main amd64 Packages
Get:1 http://archive.ubuntu.com trusty/universe amd64 Packages [5859 kB]
Get:2 http://archive.ubuntu.com trusty/main Translation-en [762 kB]
Get:3 http://archive.ubuntu.com trusty/universe Translation-en [4089 kB]

From: sudo cowbuilder --create --distribution precise --architecture amd64 --debootstrapopts --arch --debootstrapopts amd64 --basepath /var/cache/pbuilder/base-precise-amd64.cow --othermirror deb http://archive.ubuntu.com/ubuntu precise main universe:

Ign http://archive.ubuntu.com precise InRelease
Hit http://archive.ubuntu.com precise Release.gpg
Hit http://archive.ubuntu.com precise Release
Hit http://archive.ubuntu.com precise/main amd64 Packages
Get:1 http://archive.ubuntu.com precise/universe amd64 Packages [4786 kB]
Get:2 http://archive.ubuntu.com precise/main i386 Packages [1274 kB]
Get:3 http://archive.ubuntu.com precise/universe i386 Packages [4796 kB]
Get:4 http://archive.ubuntu.com precise/main TranslationIndex [3706 B]
Get:5 http://archive.ubuntu.com precise/universe TranslationIndex [2922 B]
Get:6 http://archive.ubuntu.com precise/main Translation-en [726 kB]
Get:7 http://archive.ubuntu.com precise/universe Translation-en [3341 kB]

The Precise cowbuilder --create is bringing in i386 indexes. Why?

jjekircp commented 9 years ago

I did some poking around on this and learned a few interesting things:

It seems like even on a fresh precise machine, the precise-updates repository must be enabled in sources.list to avoid the "Depends: ros-hydro-catkin but it is not going to be installed" kind of error". On my setup I added precise-updates to the repositories in the rosdistro.
The i386 indexes show up because precise has multilib (32+64 bit) support enabled by default. I found it helpful to disable this to avoid a problem with ros packages depending on python:any with apt unable to resolve the dependency. Multilib can be disabled by removing the file /etc/dpkg/dpkg.cfg.d/multilib.
Building for a different distribution requires that all the tools being used be aware of the new distribution. The chain of tools being called looks something like this: ros-deb.py -> build_binary_deb.py -> git-buildpackage -> git-pbuilder -> cowbuilder -> pbuilder -> pbuilder-buildpackage -> dpkg-buildpackage

The distribution is passed down the chain in the DIST environment variable, which pbuilder doesn't read by default. It is necessary to create /root/.pbuilderrc and add DISTRIBUTION="$DIST". Otherwise there is a line in /usr/share/pbuilder/pbuilderrc with DISTRIBUTION=trusty (assuming a trusty base system) and that's where the incorrect sources.list lines come from.

Hope this helps. I have a container on my local machine and I'm able to build both precise/hydro and trusty/indigo packages using these modifications.

mikepurvis commented 9 years ago

The instructions above from @jjekircp resolve the issue for deb builds. The final piece of this puzzle is building docs and tests for non-trusty, and the key to that is passing the DIST env var to those jobs as well, and removing sudo from the cowbuilder --execute.

Well, this has been an interesting trip down the rabbit hole.

ayoubel commented 9 years ago

@jjekircp . I have the same problem down here . i'm trying to run a fabric command to setup ros in my turtlebot2.but i've got packages building problems: Can you please publish your sources.list file for trusty? Thanks a lot

[root@devlille.iot-lab.info:2222] out: Reading state information... 0% [root@devlille.iot-lab.info:2222] out: [root@devlille.iot-lab.info:2222] out: Reading state information... Done [root@devlille.iot-lab.info:2222] out: [root@devlille.iot-lab.info:2222] out: Some packages could not be installed. This may mean that you have [root@devlille.iot-lab.info:2222] out: requested an impossible situation or if you are using the unstable [root@devlille.iot-lab.info:2222] out: distribution that some required packages have not yet been created [root@devlille.iot-lab.info:2222] out: or been moved out of Incoming. [root@devlille.iot-lab.info:2222] out: The following information may help to resolve the situation: [root@devlille.iot-lab.info:2222] out: [root@devlille.iot-lab.info:2222] out: The following packages have unmet dependencies: [root@devlille.iot-lab.info:2222] out: ros-indigo-robot-pose-publisher : Depends: libboost-system1.54.0 but it is not installable [root@devlille.iot-lab.info:2222] out: Depends: libc6 (>= 2.14) but 2.13-38+deb7u8 is to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-geometry-msgs but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-roscpp but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-tf but it is not going to be installed [root@devlille.iot-lab.info:2222] out: ros-indigo-ros-base : Depends: ros-indigo-actionlib but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-bond-core but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-class-loader but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-common-tutorials but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-dynamic-reconfigure but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-nodelet-core but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-pluginlib but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-ros-core but it is not going to be installed [root@devlille.iot-lab.info:2222] out: ros-indigo-turtlebot : Depends: ros-indigo-turtlebot-bringup but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-turtlebot-capabilities but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-turtlebot-description but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-turtlebot-teleop but it is not going to be installed [root@devlille.iot-lab.info:2222] out: ros-indigo-turtlebot-apps : Depends: ros-indigo-pano-core but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-pano-py but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-pano-ros but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-turtlebot-actions but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-turtlebot-calibration but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-turtlebot-follower but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-turtlebot-navigation but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-turtlebot-panorama but it is not going to be installed [root@devlille.iot-lab.info:2222] out: Depends: ros-indigo-turtlebot-rapps but it is not going to be installed [root@devlille.iot-lab.info:2222] out: E: Unable to correct problems, you have held broken packages. [root@devlille.iot-lab.info:2222] out:

Fatal error: sudo() received nonzero return code 100 while executing!

Requested: apt-get install -y ros-indigo-ros-base ros-indigo-turtlebot ros-indigo-robot-pose-publisher ros-indigo-turtlebot-apps python-bottle python-paste Executed: sudo -S -p 'sudo password:' /bin/bash -l -c "apt-get install -y ros-indigo-ros-base ros-indigo-turtlebot ros-indigo-robot-pose-publisher ros-indigo-turtlebot-apps python-bottle python-paste"

Aborting. Disconnecting from devlille.iot-lab.info:2222... done. sudo() received nonzero return code 100 while executing!

ayoubel commented 9 years ago

@mikepurvis could you tell me please how did you maange to rebuilde the docker container ? because i have the same kind of problem.I want to do it underlying trusty rather than precise. many thanks.

mikeferguson / buildbot-ros

Precise and Trusty on same buildbot? #46