search

APN-Pucky / pkg-testing-tools

Set of tools to simplify package testing for arch stabilizations and package bumps in Gentoo
BSD 3-Clause "New" or "Revised" License
2 stars 0 forks source link

Calculating --max-use-combinations takes.... a very long time #28

Closed juippis closed 7 months ago

juippis commented 7 months ago

To debug: pkg-testing-tool --test-feature-scope never --append-required-use '!pgo' --max-use-combinations 6 -p '=www-client/firefox-125.0.1'

If doing 1 run, with --max-use-combinations 1 it is instant. But with --max-use-combinations 6 I'm 2 hours into calculating use flag combos without result yet.

This is a familiar issue with e.g. qemu and libreoffice, but it hasn't been this "bad" with Firefox before. The use flag combo calculation has been instant. --append-required-use has no effect, I tried to remove the option and even adding !llvm_slot there.

This is on 0.2.2. Without trying, I'm sure downgrading to 0.1.2 makes it better.

juippis commented 7 months ago

Well, I did downgrade to 0.1.2 and it worked instantly :)

APN-Pucky commented 7 months ago

I received a email now :). I'll check it out. I've been running 0.2.4 with only https://github.com/APN-Pucky/pkg-testing-tools/issues/27 as a problem, so I'll fix both with 0.2.5.

APN-Pucky commented 7 months ago

Maybe I need to rewrite more of the max-use stuff given there is also https://github.com/APN-Pucky/pkg-testing-tools/issues/18 then it becomes 0.3.0. Let's see.

juippis commented 7 months ago

Then again calculating use flag combos for firefox-115.10.0 is instant too, so something in firefox-125.0.1 (probably 124+) makes the tool lose it's focus. It has at least valgrind and jumbo-build use flags, along with the new llvm_use USE_EXPAND compared to 115.10.0.

APN-Pucky commented 7 months ago

I guess I should make the 0.2.4 available on gentoo, since with that I get:

 # time pkg-testing-tool --test-feature-scope never --append-required-use '!pgo' --max-use-combinations 6 -p '=www-client/firefox-125.0.1'
[DEBUG] >>> cpv through match(): www-client/firefox-125.0.1
[INFO] >>> Following testing jobs will be executed:
=www-client/firefox-125.0.1    USE: X clang dbus -eme-free geckodriver gmp-autoupdate -hardened hwaccel -jack -jumbo-build libproxy llvm_slot_17 lto -openh264 -pgo pulseaudio -sndio -system-av1 -system-harfbuzz system-icu system-jpeg -system-libevent system-libvpx -system-png -system-python-libs system-webp telemetry valgrind wayland wifi
=www-client/firefox-125.0.1    USE: X -clang dbus -eme-free geckodriver gmp-autoupdate hardened hwaccel jack -jumbo-build -libproxy llvm_slot_17 lto -openh264 -pgo -pulseaudio -sndio -system-av1 system-harfbuzz -system-icu system-jpeg system-libevent -system-libvpx -system-png -system-python-libs -system-webp telemetry -valgrind wayland wifi
=www-client/firefox-125.0.1    USE: -X -clang dbus -eme-free -geckodriver gmp-autoupdate -hardened hwaccel -jack jumbo-build libproxy llvm_slot_17 lto -openh264 -pgo pulseaudio sndio system-av1 system-harfbuzz system-icu -system-jpeg -system-libevent -system-libvpx -system-png -system-python-libs -system-webp telemetry valgrind wayland -wifi
=www-client/firefox-125.0.1    USE: X -clang dbus -eme-free geckodriver -gmp-autoupdate hardened hwaccel -jack jumbo-build libproxy llvm_slot_17 -lto -openh264 -pgo pulseaudio -sndio -system-av1 -system-harfbuzz system-icu system-jpeg system-libevent -system-libvpx system-png -system-python-libs -system-webp telemetry -valgrind wayland -wifi
=www-client/firefox-125.0.1    USE: X clang dbus eme-free geckodriver -gmp-autoupdate hardened -hwaccel -jack -jumbo-build libproxy llvm_slot_17 -lto openh264 -pgo pulseaudio sndio -system-av1 -system-harfbuzz system-icu system-jpeg system-libevent system-libvpx -system-png -system-python-libs system-webp telemetry -valgrind wayland -wifi
=www-client/firefox-125.0.1    USE: X clang -dbus -eme-free geckodriver -gmp-autoupdate hardened -hwaccel -jack -jumbo-build -libproxy llvm_slot_17 lto openh264 -pgo pulseaudio -sndio -system-av1 system-harfbuzz system-icu -system-jpeg -system-libevent system-libvpx system-png -system-python-libs -system-webp -telemetry valgrind wayland -wifi
[INFO] >>> Running (1 of 6) =www-client/firefox-125.0.1 with USE: X clang dbus -eme-free geckodriver gmp-autoupdate -hardened hwaccel -jack -jumbo-build libproxy llvm_slot_17 lto -openh264 -pgo pulseaudio -sndio -system-av1 -system-harfbuzz system-icu system-jpeg -system-libevent system-libvpx -system-png -system-python-libs system-webp telemetry valgrind wayland wifi

These are the packages that would be merged, in order:

Calculating dependencies  ... done!
Dependency resolution took 0.24 s.

Exiting on signal 2
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.11/pkg-testing-tool", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/main.py", line 295, in main
    pkg_testing_tool(args, extra_args)
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/main.py", line 269, in pkg_testing_tool
    results.append(run_testing(job, args))
                   ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/test.py", line 107, in run_testing
    emerge_result = subprocess.run(emerge_cmdline, env=env)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 550, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1201, in communicate
    self.wait()
  File "/usr/lib/python3.11/subprocess.py", line 1264, in wait
    return self._wait(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 2051, in _wait
    (pid, sts) = self._try_wait(0)
                 ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 2009, in _try_wait
    (pid, sts) = os.waitpid(self.pid, wait_flags)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

real    0m1.720s
user    0m1.444s
sys 0m0.227s

or maybe it is some portage thing (I have 3.0.61-r1).

APN-Pucky commented 7 months ago

Oh, with 0.2.2 I also get straight to Running (1 of 6). Only idea I have left atm for that issue would be a different python version (I am on python3.11).

EDIT: On second thought maybe it is just because of some of your patches (first glimplse seems not to be the case)?

juippis commented 7 months ago

It does seem to be about the patches indeed. I should finally move all my containers to utilize profile/package.use.mask instead of pkg-testing-tool patches

0.2.2 works fine after all :) I do wonder what use flag restriction specifically is causing it, but it doesn't matter since I should move to use profiles anyway in future.

EDIT: Also weird that downgrading, with the same patch applied, does work. But oh well. Glad to see it working without them!