rpmbuild doesn't pull down Source0..SourceXX files from URL when building

[bschonec]

When building an RPM where the tarball source is located on a remote URL, the rpmbuild process in the fedora:latest comtainer can't resolve the DNS records of the URL to pull the tarball.

Example:

Source0: http://archive.apache.org/dist/ant/binaries/%{name}-%{version}-bin.tar.gz

Apache ANT doesn't come with the tarball included so the spec file must retrieve the tarball from http://archive.apache.org.

From my failed job:

Start: shell
warning: Downloading http://archive.apache.org/dist/ant/binaries/apache-ant-1.9.7-bin.tar.gz to /builddir/build/SOURCES/apache-ant-1.9.7-bin.tar.gz
curl: (6) Could not resolve host: archive.apache.org; Unknown error
error: Couldn't download http://archive.apache.org/dist/ant/binaries/apache-ant-1.9.7-bin.tar.gz
Finish: shell

The command line argument for the build is:

rpmbuild -ba --undefine=_disable_source_fetch ant.spec

This instructs rpmbuild to download all of the tarballs from "SourceX" lines in the spec file. In the spec file you can also undefine the variable _disable_source_fetch thusly:

%undefine _disable_source_fetch

I think that the ubuntu:latest runner needs to be instructed to allow the docker container 'fedora:latest' to use the runner's resolver information. If I change the Source0 entry:

Source0: http://65.108.204.189/dist/ant/binaries/%{name}-%{version}-bin.tar.gz

then the fedora:latest image does indeed download the tarball.

I don't think this is an issue with your builder but I've tried multiple things to resolve the problem without success. Perhaps your experience with the runner/podman will result in a fix.

[bschonec]

It's entirely possible that mock itself is the cause here. I tried:

mock -r fedora-rawhide-x86_64 --buildsrpm --spec ant.spec on a fedora:latest image and mock errored out with:

warning: Downloading http://archive.apache.org/dist/ant/binaries/apache-ant-1.9.7-bin.tar.gz to /builddir/build/SOURCES/apache-ant-1.9.7-bin.tar.gz
curl: (6) Could not resolve host: archive.apache.org
error: Couldn't download http://archive.apache.org/dist/ant/binaries/apache-ant-1.9.7-bin.tar.gz

yet I was able to wget the file from the same container:

[root@0da70711dffd SPECS]# wget http://archive.apache.org/dist/ant/binaries/apache-ant-1.9.7-bin.tar.gz
--2024-01-09 17:00:22--  http://archive.apache.org/dist/ant/binaries/apache-ant-1.9.7-bin.tar.gz
Resolving archive.apache.org (archive.apache.org)... 65.108.204.189, 2a01:4f9:1a:a084::2
Connecting to archive.apache.org (archive.apache.org)|65.108.204.189|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5601575 (5.3M) [application/x-gzip]
Saving to: ‘apache-ant-1.9.7-bin.tar.gz’

apache-ant-1.9.7-bin.tar.gz                       100%[============================================================================================================>]   5.34M  3.52MB/s    in 1.5s    

2024-01-09 17:00:24 (3.52 MB/s) - ‘apache-ant-1.9.7-bin.tar.gz’ saved [5601575/5601575]

So this doesn't appear to be a docker/container problem. I'll leave this open for a day or so in case you have suggestions on how to get mock to pull down the tarballs properly.

[jw3]

This should be settled with #3.

Until then you can run spectool on your own in another runner step, then map the sources into mock with the src property of this action.

e.g. something like

      - name: get sources
        run: |
          mkdir SOURCES 
          spectool -g -C SOURCES
      - uses: jw3/mock-srpm@v0
        with:
          chroot: fedora-39-x86_64
          spec: my.spec
          src: SOURCES

[bschonec]

I'm not 100% sure of what the mock config is yet, but a combination of:

config_opts['use_host_resolv'] = True
config_opts['rpmbuild_networking'] = True

in the /etc/mock/default.cfg file has enabled mock to download the externally-hosted tarball successfully. The spectool stuff probably isn't necessary. I'll find out what the minimum config is and report back.

[bschonec]

Here's the minimum that needs to happen in order for mock to download the Source0 tarball. I'll make the assumption that the SRPM or .spec file is installed at /root/rpmbuild/SPECS and that we're using fedora-39-x86_64 as the config.

# Configure the mock environment to use the host's resolver config.
cat >> /etc/mock/fedora-39-x86_64.cfg <<EOF
config_opts['use_host_resolv'] = True
config_opts['rpmbuild_networking'] = True
EOF

mock -r fedora-39-x86_64 --no-clean --buidsrpm --spec /root/rpmbuild/SPECS/ant.spec

Here's one rub, though. In the spec file I have had to:

%undefine _disable_source_fetch

which instructs rpmbuild to download the tarball. This behavior is usually the opposite where you don't want rpmbuild to download the tarball unless you give the command line:

rpmbuild -ba --undefine=_disable_source_fetch ant.spec

I'm sure there's some way we can tell mock to undefine _disable_source_fetch somehow so that the %undefine in the spec file can be remove.

Thoughts?

[jw3]

I haven't take a close look yet, but the thoughts behind #3 are that you really shouldn't care if network is enabled or not, the action provides the sources (so they can be cached).

Over in the RPM build action it's also going to default to no network, that's how a Koji build runs. But, Copr has a flag to enable networking, the same could be done for this and the RPM action.

[jw3]

@bschonec Fetching remote sources is now possible in v1 by setting fetch-sources: true.

See #14

[bschonec]

This is a wonderful upgrade! Thanks!