Support for running `tmt` in `tmt`

[psss]

There are use cases when a nested instance of tmt should be run under the parent instance. For example, reserving a bare metal host as a hypervisor and running several virtual machines as guests there. Currently it is possible but there is no support for test results aggregation from the child processes. It would be good to provide a comfortable way to see test results from the child plans as well.

Also, an alternative approach was proposed by @kkaarreell today: The provision step phases describing the child guests would be included in a single provision step config together with the phase describing the host. Once the guests would be started, their ssh connection details would be handed over to the parent tmt process.

There is also this idea of a tmt go command which would generate relevant combinations of environment dimensions, then schedule individual tmt run commands to execute tests for each supported combination and finally gather the test results from all. This approach seems to be slightly consistent with the first option above.

[happz]

Also, an alternative approach was proposed by @kkaarreell today: The provision step phases describing the child guests would be included in a single provision step config together with the phase describing the host. Once the guests would be started, their ssh connection details would be handed over to the parent tmt process.

This sums up the idea of "chained provisioning", e.g. how: beaker followed by a dozen of how: virtual on that guest. Which I liked at first, but after thinking about it during the meeting, I realized there were some complications. Two assrted thoughts:

• The provisioning itself, plugins spawning VMs - libvirt, virtual.testcloud - operate on localhost by default, so far. We would need to extend them to run on a given guest, performing their actions remotely.
• Making these remote VMS - "second-level guests" - accessible via SSH might require a nontrivial setup on their host, which, given the many use cases mentioning network topology, might be very prone to various issues, dropped connections, and general instability given what tests will torture these guests and their network setups. On the other hand, working with them locally, from the same machine, might make channels easier to establish and easier to preserve (e.g. a direct console rather than SSH over network the test modifies).

[kkaarreell]

Also, an alternative approach was proposed by @kkaarreell today: The provision step phases describing the child guests would be included in a single provision step config together with the phase describing the host. Once the guests would be started, their ssh connection details would be handed over to the parent tmt process.

TBH, I wasn't thinking about provisioning those virtual guests by tmt. I thought that would be a responsibility of the "setup task" executed on the host. My understanding was that the purpose is to test various configurations of the virtual guest and I thought that in general tmt would be behind with features/specification compared to the actual development/implementation of the virtualization.

[kkaarreell]

* Making these remote VMS - "second-level guests" - accessible via SSH might require a nontrivial setup on their host, which, given the many use cases mentioning network topology, might be very prone to various issues, dropped connections, and general instability given what tests will torture these guests and their network setups. On the other hand, working with them locally, from the same machine, might make channels easier to establish and easier to preserve (e.g. a direct console rather than SSH over network the test modifies).

I am not saying this can't be true, but if I am not mistaken, for the minimal implementation we are speaking about a single ssh tunnel enabling tmt to connect to the guest.

[happz]

• Making these remote VMS - "second-level guests" - accessible via SSH might require a nontrivial setup on their host, which, given the many use cases mentioning network topology, might be very prone to various issues, dropped connections, and general instability given what tests will torture these guests and their network setups. On the other hand, working with them locally, from the same machine, might make channels easier to establish and easier to preserve (e.g. a direct console rather than SSH over network the test modifies).

I am not saying this can't be true, but if I am not mistaken, for the minimal implementation we are speaking about a single ssh tunnel enabling tmt to connect to the guest.

True, but it would have to stay up and running no matter what happens to the network stack on the host and guest. I mean, I don't know, maybe it's a moot point, I just heard users speaking about bridges and network interfaces and their setup so much that it got me worried that preserving the channel to get necessary info, especially when the test fails, might be way too much work :)