1.33 regression: `dist-git-source` behaves differently in different provisions

[LecrisUT]

The issue occurs in https://src.fedoraproject.org/rpms/python-scikit-build-core/pull-request/59

There I am using:

discover:
  how: fmf
  dist-git-source: true
  dist-git-extract: scikit_build_core-*/
/example:
  summary:
    Documentation examples
  discover+:
    how: fmf
    filter: "tag: examples"
    path: docs
  execute:
    how: tmt

In 1.32 (2 weeks ago) the tests were still ok, but now I am getting issues that the tmt tree is not found:

output

```console /plans/examples summary: Documentation examples discover how: fmf directory: docs Tests will be discovered after dist-git patching in prepare. summary: 0 tests selected provision queued provision.provision task #1: default-0 provision.provision task #1: default-0 how: connect primary address: 3.17.12.31 topology address: 3.17.12.31 port: 22 key: /etc/citool.d/id_rsa_artemis multihost name: default-0 arch: x86_64 distro: Fedora Linux 41 (Cloud Edition Prerelease) kernel: 6.9.0-64.fc41.x86_64 package manager: dnf5 selinux: yes is superuser: yes summary: 1 guest provisioned prepare queued push task #1: push to default-0 push task #1: push to default-0 queued prepare task #1: Prepare dist-git sources (buildrequires, patches, discovery...) on default-0 queued prepare task #2: requires (dist-git) on default-0 queued prepare task #3: requires on default-0 queued prepare task #4: recommends (dist-git) on default-0 prepare task #1: Prepare dist-git sources (buildrequires, patches, discovery...) on default-0 how: distgit name: Prepare dist-git sources (buildrequires, patches, discovery...) path: docs fail: Metadata tree path 'docs' not found. finish summary: 0 tasks completed plan failed The exception was caused by 1 earlier exceptions Cause number 1: prepare step failed The exception was caused by 1 earlier exceptions Cause number 1: Metadata tree path 'docs' not found. ```

What is confusing is that when I try it locally with provision.how = container and the tmt-reproducer, this seems to go through:

provision container

```console $ tmt -c arch=x86_64 -c distro=fedora-rawhide -c initiator=fedora-ci -c trigger=commit run -a --verbose -e @tmt-environment-plans-examples.yaml provision --how container --image fedora:rawhide plan --name ^/plans/examples$ prepare queued push task #1: push to default-0 push task #1: push to default-0 queued prepare task #1: Prepare dist-git sources (buildrequires, patches, discovery...) on default-0 queued prepare task #2: requires (dist-git) on default-0 queued prepare task #3: requires on default-0 queued prepare task #4: recommends (dist-git) on default-0 prepare task #1: Prepare dist-git sources (buildrequires, patches, discovery...) on default-0 how: distgit name: Prepare dist-git sources (buildrequires, patches, discovery...) path: docs filter: tag: examples summary: 7 tests selected /examples/downstream/pybind11_example /examples/getting_started/abi3 /examples/getting_started/c /examples/getting_started/cython /examples/getting_started/fortran /examples/getting_started/pybind11 /examples/getting_started/swig ```

I would try the full reproducer but I am getting another issue which might be unrelated (ssh timeout): #2937. After managing to make it work on F39/F40, I am indeed reproducing the issue with provision.how: virtual

Update: Seems like F40 PR works, something weird is going on. Sort of works. It worked on Zuul, but failed on Fedora-CI. Might be having a flaky response. This seems to be very flaky when it works or not :worried:

[LecrisUT]

Could be related to RPM 4.20 :thinking:

[LecrisUT]

Ok, so in 1.34 this issue is resolved (afaiu it was RPM 4.20 issue), but then there is a more specific issue with the tmt-reproducer where the dist-git prepare step is not performed when it is split up in steps:

Output

```console $ tmt --root . -c arch=x86_64 -c distro=fedora-rawhide -c trigger=commit run --until provision --verbose -e @tmt-environment-plans-examples.yaml provision --how virtual --image Fedora-Rawhide plan --name ^/plans/examples$ /var/tmp/tmt/run-019 Found 1 plan. /plans/examples summary: Documentation examples discover how: fmf directory: docs warn: Sources will not be extracted, prepare step is not enabled. Tests will be discovered after dist-git patching in prepare. summary: 0 tests selected provision queued provision.provision task #1: default-0 $ tmt --root . run --last --since prepare /var/tmp/tmt/run-019 /plans/examples prepare queued push task #1: push to default-0 push task #1: push to default-0 queued prepare task #1: requires on default-0 prepare task #1: requires on default-0 how: install summary: Install required packages name: requires where: default-0 package: /usr/bin/flock queued pull task #1: pull from default-0 pull task #1: pull from default-0 summary: 1 preparation applied execute ```

@happz @lukaszachy Any ideas what caused this regression? Afaict https://github.com/teemtee/tmt/pull/2641 caused this workflow to break. Seems like the prepare steps are not saved when running the discover step (or rather when running every phase)

[lukaszachy]

Discover prints this warn: Sources will not be extracted, prepare step is not enabled.

There is no way to extract sources now without running the provision & prepare steps. With your rfe #2994 it might be possible, I'm not sure if the 'prepare.distgit' isn't enabled only in runtime.

How big problem is this for you?

[LecrisUT]

How big problem is this for you?

Only affects tmt-reproducer so not urgent for me, but others trying to run this might find this surprise and be less clear why it doesn't work.

One patch could be to instead of splitting the tmt-reproducer steps, to add the guest-setup-0.sh as a prepare step, e.g.

$ tmt run prepare --insert --how shell --script "./guest-setup-0.sh"

It seems to work on my side, but not sure about what order it should have. @thrix thoughts?

There is no way to extract sources now without running the provision & prepare steps. With your rfe https://github.com/teemtee/tmt/issues/2994 it might be possible, I'm not sure if the 'prepare.distgit' isn't enabled only in runtime.

The issue is rather that the prepare step is not saved, e.g. in run-xxx/**/prepare/step.yaml it only contains a prepare.shell step (not sure why, would have expected an install) after the first half command is executed.

[lukaszachy]

AFAIK the 'install' step injection (require, recommend) happens as 'runtime' only.