search

ros-infrastructure / ros_release_python

3 stars 4 forks source link

debian generators do not leverage existing orig.tar.gz files #32

Open tfoote opened 5 years ago

tfoote commented 5 years ago

Every time you rerun the generator it regenerates the orig.tar.gz. This means that if you don't upload everything from the same build you will get conflicting files in the bootstrap repository which will fail on aptly export and requires manual clearing of the files from the aptly repository.

It would be valuable if we reused existing orig.tar.gz files that are local as a first step. And as a second step it would fetch the sources from the repository if they already exists. Otherwise it's basically impossible to do a debian increment.

dirk-thomas commented 5 years ago

This means that if you don't upload everything from the same build ...

Can you clarify in which scenario this is the case.

tfoote commented 5 years ago

I ran it without the upload argument for the deb3 command, checked on the outputs. Then copy and pasted the dput command to run the upload. Then I ran the pip build and ran it's upload. But I realized I'd failed to upload the deb and had only run the dsc upload command, but not the second deb command. So I went back to run the deb upload, which because it had cleared the artifacts during the pip build I had to rebuild the deb3. And that follow on upload imported, but then failed to publish due to the conflicting orig.tar.gz.

After that I tried to do debian increment releases, but every attempt to upload at that point failed, because aptly can no longer publish because it has references to both versions of the orig.tar.gz. This blocked all my follow on attempted releases, including trying a full patch version bump. It would have blocked any future release as well until I went in and cleared the conflicting packages from the aptly database.

Note that for the debian increment releases I even tried manually downloading the orig.tar.gz locally using apt-get source to see if it would reuse the tarball but it regenerated anyway.

dirk-thomas commented 5 years ago

I ran it without the upload argument for the deb3 command, checked on the outputs. Then copy and pasted the dput command to run the upload. Then I ran the pip build and ran it's upload. But I realized I'd failed to upload the deb and had only run the dsc upload command, but not the second deb command. So I went back to run the deb upload, which because it had cleared the artifacts during the pip build I had to rebuild the deb3. And that follow on upload imported, but then failed to publish due to the conflicting orig.tar.gz.

I am not sure it makes sense to spend effort to support this pretty awkward manual workflow.

Note that for the debian increment releases I even tried manually downloading the orig.tar.gz locally using apt-get source to see if it would reuse the tarball but it regenerated anyway.

That limitation seems to be similar to our other toolchains which can't regenerate the source package without a diff on a deb inc. We commonly work around it by just releasing new version instead of a deb inc in the rate case where we only change Debian specific metadata.