Handle patches

[ralphbean]

We got this error:

 [2014-11-21 13:12:23][    fedmsg    INFO] Running u'rpmbuild -bs /var/tmp/thn-ecJi3O/opendkim.spec' in '/var/tmp/thn-ecJi3O'
 [2014-11-21 13:12:23][    fedmsg WARNING] error: Bad source: ./opendkim.keygen-permissions.patch: No such file or directory
 [2014-11-21 13:12:23][    fedmsg   ERROR] return code 1
 [2014-11-21 13:12:24][    fedmsg   ERROR] {'body': {u'username': u'root', u'certificate': u'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVZRENDQThtZ0F3SUJBZ0lDQWRBd0R...VhNQlVHQTFVRUNoT
 Q4OGlFSk5zaGxKS3FVdlNETG5ldXFSNlV4UGQKRGZXekUzZGpVeUppWm0rM3lwZ2ZYdGZG\nSTFoWXJFZ2RUM2lRYW5PRzM1dDBUYjFuU05DSWxPZGR2ZFBaTnVjOQpxNVNURlFPUzhkNnBjY2Vj\nQnlUellMd01kY...2-90a0-cdf2cec95
 Traceback (most recent call last):
 File "/usr/lib/python2.7/site-packages/moksha/hub/api/consumer.py", line 190, in _work
 self.consume(message)
 File "/usr/lib/python2.7/site-packages/hotness/consumers.py", line 108, in consume
 self.handle_anitya(msg)
 File "/usr/lib/python2.7/site-packages/hotness/consumers.py", line 162, in handle_anitya
 task_id = self.buildsys.handle(package, upstream, version, bz)
 File "/usr/lib/python2.7/site-packages/hotness/buildsys.py", line 108, in handle
 output = self.run(['rpmbuild', '-bs', specfile], cwd=tmp)
 File "/usr/lib/python2.7/site-packages/hotness/buildsys.py", line 75, in run
 raise Exception
 Exception

We need to be able to download patches -- is this one from dist-git?

Also, on failures like this we should publish a failure fedmsg.

[bochecha]

$ ls
opendkim.spec  sources
$ cat sources 
80043a5147ae4ff2feb7f8d32ed2ccac  opendkim.autocreate-keys-no.patch
a6a9c36328a9b4c18f5fca19e3b1aabc  opendkim.keygen-permissions.patch
4f9d50fcd8428c280b2a4739cf4b8492  opendkim.systemd-no-default-genkey.patch
08cc80a2aedec62b0444d8d6af24a155  opendkim-2.9.2.tar.gz

The patch should be downloaded with the tarball, automatically, by fedpkg sources. Did that not work?

[pypingou]

https://github.com/fedora-infra/the-new-hotness/blob/develop/hotness/buildsys.py#L91 I guess we should use sh.fedpkg.clone instead of clone directly via git then

[bochecha]

@pypingou Not at all, there's no reason to do that

What you need is to replace this line :

output = self.run(['spectool', '-g', specfile], cwd=tmp)

By a call to fedpkg sources instead.

spectool only downloads what is declared as SourceX, which is not always the same as what was uploaded to the lookaside cache. For example, some source files are text only, and small enough that it makes more sense keeping them in dist-git, whereas some patches are so big (and sometimes compressed) that they should be uploaded to the lookaside cache.

On the other hand, fedpkg sources doesn't care at all about SourceX or PatchX, it merely looks at the sources file, and downloads it all from the lookaside cache.

[ralphbean]

The patch should be downloaded with the tarball, automatically, by fedpkg sources. Did that not work?

Yup - it actually used to use fedpkg srpm which did this correctly, but long story short, I removed it to try and make my life easier in stg. Using fedpkg sources is the right thing to do, yes.

We'll need to locally configure fedpkg in stg (via /etc/rpkg/... or whatever the path is) to get files from pkgs01.phx2.fedoraproject.org instead of pkgs.fedoraproject.org -- which is one of the problems I ran into earlier in the week. /etc/hosts is another option.

[bochecha]

Here's a trick:

# ln -s /usr/bin/fedpkg /usr/bin/fedpkgstg
# cat /etc/rpkg/fedpkg.conf 
[fedpkg]
lookaside = http://pkgs.fedoraproject.org/repo/pkgs
lookasidehash = md5
lookaside_cgi = https://pkgs.fedoraproject.org/repo/pkgs/upload.cgi
gitbaseurl = ssh://%(user)s@pkgs.fedoraproject.org/%(module)s
anongiturl = git://pkgs.fedoraproject.org/%(module)s
tracbaseurl = https://%(user)s:%(password)s@fedorahosted.org/rel-eng/login/xmlrpc
branchre = f\d$|f\d\d$|el\d$|olpc\d$|master$
kojiconfig = /etc/koji.conf
build_client = koji

[fedpkgstg]
lookaside = http://pkgs.stg.fedoraproject.org/repo/pkgs
lookasidehash = md5
lookaside_cgi = https://pkgs.stg.fedoraproject.org/repo/pkgs/upload.cgi
gitbaseurl = ssh://%(user)s@pkgs.stg.fedoraproject.org/%(module)s
anongiturl = git://pkgs.stg.fedoraproject.org/%(module)s
tracbaseurl = https://%(user)s:%(password)s@fedorahosted.org/rel-eng/login/xmlrpc
branchre = f\d$|f\d\d$|el\d$|olpc\d$|master$
kojiconfig = /etc/koji.conf
build_client = koji

Then, use the new fedpkgstg command:

$ fedpkgstg -v clone ipset
Cloning ssh://bochecha@pkgs.stg.fedoraproject.org/ipset
Running git clone ssh://bochecha@pkgs.stg.fedoraproject.org/ipset --origin origin directly on the tty
Cloning into 'ipset'...
remote: Counting objects: 105, done.
remote: Compressing objects: 100% (102/102), done.
remote: Total 105 (delta 40), reused 0 (delta 0)
Receiving objects: 100% (105/105), 18.68 KiB | 0 bytes/s, done.
Resolving deltas: 100% (40/40), done.
Checking connectivity... done.

[ralphbean]

Fancy!