search

fedora-copr / copr

RPM build system - upstream for https://copr.fedorainfracloud.org/
111 stars 57 forks source link

high-performance builder #3267

Open spotrh opened 2 months ago

spotrh commented 2 months ago

My texlive builds are failing because they're taking up more disk space than the copr builder has. It was suggested that a high-performance builder might have more storage, if so, please enable them for the following regex:

@spot/texlive-2024-testing/.(x86_64|aarch64|ppc64le|s390x)/texlive*

xsuchy commented 2 months ago

Just being curious I checked the log (the build is https://copr.fedorainfracloud.org/coprs/spot/texlive-2024-testing/build/7460130/ ) and the last lines of 2.9GB big log file is:

...
Wrote: /builddir/build/RPMS/texlive-collection-plaingeneric-svn68675-72.fc40.noarch.rpm
Wrote: /builddir/build/RPMS/texlive-gloss-occitan-svn52593-72.fc40.noarch.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.1H0xM6
+ umask 022
+ cd /builddir/build/BUILD
+ cd texlive-2024
+ /usr/bin/rm -rf /builddir/build/BUILDROOT/texlive-2024-72.fc40.x86_64
+ RPM_EC=0
++ jobs -p
+ exit 0
Executing(rmbuild): /bin/sh -e /var/tmp/rpm-tmp.GyLNzD
+ umask 022
+ cd /builddir/build/BUILD
+ rm -rf /builddir/build/BUILD/texlive-2024-SPECPARTS
+ rm -rf texlive-2024 texlive-2024.gemspec
+ RPM_EC=0
++ jobs -p
+ exit 0
Finish: rpmbuild texlive-2024-72.fc40.src.rpm
Finish: build phase for texlive-2024-72.fc40.src.rpm
INFO: chroot_scan: 1 files copied to /var/lib/copr-rpmbuild/results/chroot_scan
INFO: /var/lib/mock/fedora-40-x86_64-1716086259.668078/root/var/log/dnf5.log
ERROR: Exception(/var/lib/copr-rpmbuild/results/texlive-2024-72.fc40.src.rpm) Config(fedora-40-x86_64) 137 minutes 18 seconds
INFO: Results and/or logs in: /var/lib/copr-rpmbuild/results
INFO: Cleaning up build root ('cleanup_on_failure=True')
Start: clean chroot
INFO: unmounting tmpfs.
Finish: clean chroot
Traceback (most recent call last):
  File "/usr/libexec/mock/mock", line 1095, in <module>
    exitStatus = main()
                 ^^^^^^
  File "/usr/lib/python3.12/site-packages/mockbuild/trace_decorator.py", line 93, in trace
    result = func(*args, **kw)
             ^^^^^^^^^^^^^^^^^
  File "/usr/libexec/mock/mock", line 852, in main
    result = run_command(options, args, config_opts, commands, buildroot)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mockbuild/trace_decorator.py", line 93, in trace
    result = func(*args, **kw)
             ^^^^^^^^^^^^^^^^^
  File "/usr/libexec/mock/mock", line 969, in run_command
    mockbuild.rebuild.do_rebuild(config_opts, commands, buildroot, options, srpms)
  File "/usr/lib/python3.12/site-packages/mockbuild/trace_decorator.py", line 93, in trace
    result = func(*args, **kw)
             ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mockbuild/rebuild.py", line 85, in do_rebuild
    rebuild_generic(srpms, commands, buildroot, config_opts, cmd=build,
  File "/usr/lib/python3.12/site-packages/mockbuild/trace_decorator.py", line 93, in trace
    result = func(*args, **kw)
             ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mockbuild/rebuild.py", line 25, in rebuild_generic
    ret = cmd(item)
          ^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mockbuild/rebuild.py", line 66, in build
    commands.build(srpm, timeout=config_opts['rpmbuild_timeout'],
  File "/usr/lib/python3.12/site-packages/mockbuild/trace_decorator.py", line 93, in trace
    result = func(*args, **kw)
             ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mockbuild/backend.py", line 326, in build
    self.build_results.extend(self.copy_build_results(results))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mockbuild/trace_decorator.py", line 93, in trace
    result = func(*args, **kw)
             ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mockbuild/backend.py", line 824, in copy_build_results
    shutil.copy2(item, self.buildroot.resultdir)
  File "/usr/lib64/python3.12/shutil.py", line 463, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib64/python3.12/shutil.py", line 273, in copyfile
    _fastcopy_sendfile(fsrc, fdst)
  File "/usr/lib64/python3.12/shutil.py", line 164, in _fastcopy_sendfile
    raise err from None
  File "/usr/lib64/python3.12/shutil.py", line 150, in _fastcopy_sendfile
    sent = os.sendfile(outfd, infd, offset, blocksize)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 28] No space left on device: '/var/lib/mock/fedora-40-x86_64-1716086259.668078/root/builddir/build/RPMS/texlive-shobhika-svn50555-72.fc40.noarch.rpm' -> '/var/lib/copr-rpmbuild/results/texlive-shobhika-svn50555-72.fc40.noarch.rpm'
Copr build error: Build failed

It fails in very last moment of build, so it did not fit just by a thin hair.

To anticipate any future builds I suggest using:

@spot/texlive.*/.*/texlive*

praiskup commented 2 months ago

To anticipate any future builds I suggest using: @spot/texlive././texlive*

There are no powerful ppc64le builders, though. No builder would be found for such a task, and that's why it would stay in the starting state forever.

xsuchy commented 2 months ago

Unfortunately this cannot be done by just changing configuration, we will have to do a coding first (https://github.com/fedora-copr/copr/issues/3268). I expect ETA of several weeks (cca 2 months). :(