ros-infrastructure / buildfarm

Build scripts and notes for catkin debian build pipeline.
6 stars 15 forks source link

deb inc sourcedebs fail to upload the first time #115

Closed tfoote closed 10 years ago

tfoote commented 11 years ago

This is due to an md5sum collision on the orig.tar.gz The correct solution is to create a deb with a diff to the orig.tar.gz to upload instead of uploading a new orig tarball.

As a work around a second run of the sourcedeb usually works.

running command ['reprepro', '-v', '-b', '/var/www/repos/building', 'include', 'lucid', '/tmp/upload/ros-fuerte-ros-comm_lucid/lucid/ros-fuerte-ros-comm_1.8.16-1lucid_source.changes']
File "pool/main/r/ros-fuerte-ros-comm/ros-fuerte-ros-comm_1.8.16.orig.tar.gz" is already registered with different checksums!
md5 expected: a03a1c51d2bb8c9ed8ed20a63f6d0343, got: 6a67879d41a57260796dd2815fdd8fe0
sha1 expected: 2874227147d2e7fd42b34dd4ea8f95229c7a6cf4, got: 2c0b5aa7688de8bec1e001f5dd6bc579bff4f9ca
sha256 expected: 2c4b63bbf966e1762eddfc55cc23bd7da2343b9c927f05c3131f2a9a2462abd7, got: 701609b904a88471b5932e8976462683e833521fc186aa18547c728c3cb3b3bf
size expected: 948974, got: 948207
There have been errors!
tfoote commented 11 years ago

Apparently reprepro was keeping the unreferenced files around after removing them. Now the invalidation, both removes the files from the distro and then clears all unreferenced. 4a002854cc5e69a2c83e4a2deb3124dbdbebb3f3

tfoote commented 11 years ago

not fixed: http://jenkins.willowgarage.com:8080/job/ros-hydro-pcl_sourcedeb/34/console

tfoote commented 11 years ago

The sourcedebs need to clear all architectures at once instead of one at a time since they all share the same sourcedeb.

tfoote commented 11 years ago

working on the sync script here: https://github.com/ros-infrastructure/reprepro-updater/commit/733ff6de70e9e7e707681d06cc603dbb91c76793

tfoote commented 10 years ago

With the new patch above and a slight fix to it, a new sourcedeb job removes the orig.tar.gz before uploading the new one.

running ['reprepro', '-b', '/var/www/repos/building', '-T', 'dsc', '-V', 'removefilter', u'precise', 'Package (== ros-hydro-serial )']
removing 'ros-hydro-serial' from 'precise|main|source'...
Exporting indices...
Successfully created '/var/www/repos/building/dists/precise/Release.gpg.new'
Successfully created '/var/www/repos/building/dists/precise/InRelease.new'
Deleting files no longer referenced...
deleting and forgetting pool/main/r/ros-hydro-serial/ros-hydro-serial_1.1.4-7precise.debian.tar.gz
deleting and forgetting pool/main/r/ros-hydro-serial/ros-hydro-serial_1.1.4-7precise.dsc
running ['reprepro', '-b', '/var/www/repos/building', '-T', 'dsc', '-V', 'removefilter', u'quantal', 'Package (== ros-hydro-serial )']
removing 'ros-hydro-serial' from 'quantal|main|source'...
Exporting indices...
Successfully created '/var/www/repos/building/dists/quantal/Release.gpg.new'
Successfully created '/var/www/repos/building/dists/quantal/InRelease.new'
Deleting files no longer referenced...
deleting and forgetting pool/main/r/ros-hydro-serial/ros-hydro-serial_1.1.4-7quantal.debian.tar.gz
deleting and forgetting pool/main/r/ros-hydro-serial/ros-hydro-serial_1.1.4-7quantal.dsc
running ['reprepro', '-b', '/var/www/repos/building', '-T', 'dsc', '-V', 'removefilter', u'raring', 'Package (== ros-hydro-serial )']
removing 'ros-hydro-serial' from 'raring|main|source'...
Exporting indices...
Successfully created '/var/www/repos/building/dists/raring/Release.gpg.new'
Successfully created '/var/www/repos/building/dists/raring/InRelease.new'
Deleting files no longer referenced...
deleting and forgetting pool/main/r/ros-hydro-serial/ros-hydro-serial_1.1.4-7raring.debian.tar.gz
deleting and forgetting pool/main/r/ros-hydro-serial/ros-hydro-serial_1.1.4-7raring.dsc
deleting and forgetting pool/main/r/ros-hydro-serial/ros-hydro-serial_1.1.4.orig.tar.gz
running ['reprepro', '-v', '-b', '/var/www/repos/building', 'deleteunreferenced']

This should fix this issue. I'll close when I see a new debinc come in from non test repositories.

tfoote commented 10 years ago

It worked for hrpsys-0.0.1-7 w/o two tries! http://jenkins.ros.org/job/ros-groovy-hrpsys_sourcedeb/