Trying to emerge UNKEYWORDED packages results in IndexError: list index out of range

[juippis]

Hey,

trying to emerge sys-power/nut-9999 gives me:

# pkg-testing-tool --append-emerge '--autounmask=y --oneshot' --extra-env-file 'test.conf' --max-use-combinations 6 --test-feature-scope once -p '=sys-power/nut-9999'
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.11/pkg-testing-tool", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/main.py", line 408, in main
    pkg_testing_tool(args, extra_args)
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/main.py", line 344, in pkg_testing_tool
    for new_job in define_jobs(atom, args):
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/main.py", line 236, in define_jobs
    package_metadata = get_package_metadata(atom)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/main.py", line 123, in get_package_metadata
    cpv = atom_to_cpv(atom)
          ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/use.py", line 59, in atom_to_cpv
    return portage.db[portage.root]['porttree'].dbapi.match(atom)[0]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Downgrading to 0.1.2 works fine. I can only guess 0666ce82fea1a55c95367a39d0a65dba6f8277bd is the culprit. Not sure if this affects every *-9999 package, but "normal" releases seem to still work just fine.

I saw you made a new release, haven't tried with that yet.

[juippis]

An update, this seems to be generally whenever trying to emerge package without KEYWORDS:

# pkg-testing-tool --debug -p '=mate-base/mate-panel-1.27.3'
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.11/pkg-testing-tool", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/main.py", line 476, in main
    pkg_testing_tool(args, extra_args)
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/main.py", line 403, in pkg_testing_tool
    for new_job in define_jobs(atom, args):
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/main.py", line 293, in define_jobs
    package_metadata = get_package_metadata(atom)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/main.py", line 169, in get_package_metadata
    cpv = atom_to_cpv(atom)
          ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pkg_testing_tools/use.py", line 76, in atom_to_cpv
    return portage.db[portage.root]["porttree"].dbapi.match(atom)[0]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

[juippis]

Also I'm getting this on stable, when trying to keyword ~unstable packages. So somehow it's not handling packages with "keywords-above" very well anymore. You should be able to test ~unstable packages on stable system to keep stabilizing them.

[APN-Pucky]

Sorry I somehow missed this issue, before (also was away...). In my experience this error is cause by typos/missing packages. I just updated it to make the error slightly clearer: https://github.com/APN-Pucky/pkg-testing-tools/commit/066f2106097ecad520ee37dc9622d0c8e6b42531#diff-28d1d9af37bef7ad3dea368b0fd1add4e29eb174f2c76eff791fb5b6ac958e51 (i'll also add a print of the atom to the error) [tbc.]

[APN-Pucky]

@juippis I think the solution to your problem might be that the 9999 package is masked: This fails

$ pkg-testing-tool -p '=sci-physics/yoda-9999'

while this works for me (approx using current git version):

$ ACCEPT_KEYWORDS="**" pkg-testing-tool -p '=sci-physics/yoda-9999'

[juippis]

Yes, but the tool was originally created to test keywording/stabilization, so it should automatically handle "non-visible" keywords on a package. This is especially annoying on stable system, when trying to stabilize ~unstable packages :P

[APN-Pucky]

The change you mentioned https://github.com/APN-Pucky/pkg-testing-tools/commit/0666ce82fea1a55c95367a39d0a65dba6f8277bd was suggested by Zac https://bugs.gentoo.org/918693 in order to support ~app/foo-x.y.z-r1 revision matching. I'll see how it works with unmasking live/9999. For normal unstable unmasking I use --append-emerge "--autounmask=y --autounmask-write --autounmask-continue --autounmask-use=y", but that does not seem to work for 9999 straightforwardly.

[APN-Pucky]

I think the best/quickest solution here is to use the old behavior for 9999 builds: https://github.com/APN-Pucky/pkg-testing-tools/commit/b4fa82b2c97fedd0aaf18112b40d90f2fa0f2144. Or should it also work like this for ~unstable packages, because then revisions/atoms beyond '=' won't work again. Please let me know if this is again how you expect it to behave. Should even be easy to test once https://github.com/gentoo/gentoo/pull/35388 is merged.

[APN-Pucky]

I think to also "restore" older behavior for ~unstable I'll merge this PR https://github.com/APN-Pucky/pkg-testing-tools/pull/14, which implements:

1. Try to match the atom, only tests unmasked packages, but supports -rN, * etc..
2. Otherwise use old behavior, i.e. assume that package exists and can be unmasked

Alternatively, I could probably inject the accept_keywords just for the matching in 1., but that seems a bit ugly.

[APN-Pucky]

Injecting ACCEPT_KEYWORDS for 1. seems to work too, but I am not sure about side effects: https://github.com/APN-Pucky/pkg-testing-tools/pull/15/commits/86e6595dd508018e8df69db7f31572a1d1695f5d This would be best IMO if there are no side effects since it also supports revisions ~ and ignores missing keywords.

[juippis]

Thanks for working on this!