tmt clean guests|runs ... doesn't

martinpitt commented 8 months ago

With tmt-1.31.0-1.fc39.noarch, tmt clean doesn't seem to do anything. I see this with sessions like tmt run --until report provision --how virtual --image fedora-39 (I do this to interactively debug after a failure), but a session can also be generated with tmt try -l -v fedora-rawhide (which apparently fails to boot and times out trying to connect after two minutes). After that:

❱❱❱ virsh list --all
 Id   Name               State
----------------------------------
 2    tmt-001-YTPzyviG   running

❱❱❱ ls -ld /var/tmp/tmt/run* /var/tmp/tmt/testcloud/instances/*
drwxr-xr-x. 3 martin martin 4096 Mar 20 06:42 /var/tmp/tmt/run-001
drwxr-xr-x. 3 martin martin 4096 Mar 20 06:42 /var/tmp/tmt/testcloud/instances/tmt-001-YTPzyviG

After that, neither of these commands has any observable effect, both the libvirt VM and the run- and instances/ remain untouched:

❱❱❱ tmt clean guests -vvdd
clean
guests

❱❱❱ tmt clean runs -vvdd
clean
runs

The output looks a bit different with -l/--last:

❱❱❱ tmt clean runs -vvddl
clean
runs
Workdir '/var/tmp/tmt/run-001' already exists.
    Removing workdir '/var/tmp/tmt/run-001'.

❱❱❱ tmt clean guests -vvddl
clean
guests
No metadata found, using the default plan.
Workdir '/var/tmp/tmt/run-001' created.
Read file '/var/tmp/tmt/run-001/run.yaml'.
Run data not found.
Workdir '/var/tmp/tmt/run-001/default/plan' created.
Create the data directory '/var/tmp/tmt/run-001/default/plan/data'.
Create the environment file '/var/tmp/tmt/run-001/default/plan/data/variables.env'.
    Workdir '/var/tmp/tmt/run-001/default/plan/discover' created.
    Read file '/var/tmp/tmt/run-001/default/plan/discover/step.yaml'.
    Step data not found.
    Read file '/var/tmp/tmt/run-001/default/plan/discover/tests.yaml'.
    Discovered tests not found.
    Workdir '/var/tmp/tmt/run-001/default/plan/provision' created.
    Read file '/var/tmp/tmt/run-001/default/plan/provision/step.yaml'.
    Step data not found.
    Read file '/var/tmp/tmt/run-001/default/plan/provision/guests.yaml'.
    Provisioned guests not found.
    Workdir '/var/tmp/tmt/run-001/default/plan/prepare' created.
    Read file '/var/tmp/tmt/run-001/default/plan/prepare/step.yaml'.
    Step data not found.
    Workdir '/var/tmp/tmt/run-001/default/plan/execute' created.
    Read file '/var/tmp/tmt/run-001/default/plan/execute/step.yaml'.
    Step data not found.
    Read file '/var/tmp/tmt/run-001/default/plan/execute/results.yaml'.
    Test results not found.
    Workdir '/var/tmp/tmt/run-001/default/plan/report' created.
    Read file '/var/tmp/tmt/run-001/default/plan/report/step.yaml'.
    Step data not found.
    Workdir '/var/tmp/tmt/run-001/default/plan/finish' created.
    Read file '/var/tmp/tmt/run-001/default/plan/finish/step.yaml'.
    Step data not found.

But same result, no actual cleaning.

skycastlelily commented 8 months ago

Able to reproduce this problem if I run clean runs first,but if I run clean guests first,all things work well for testcloud created guests.

lnie@fedora:~/test-tmt$ tmt clean guests -vvddl
clean
guests
Using tree '/home/lnie/test-tmt'.
Workdir '/var/tmp/tmt/run-025' already exists.
Read file '/var/tmp/tmt/run-025/run.yaml'.
    Workdir '/var/tmp/tmt/run-025/plans/testcloud' already exists.
    Create the data directory '/var/tmp/tmt/run-025/plans/testcloud/data'.
    Create the environment file '/var/tmp/tmt/run-025/plans/testcloud/data/variables.env'.
        Workdir '/var/tmp/tmt/run-025/plans/testcloud/discover' already exists.
        Read file '/var/tmp/tmt/run-025/plans/testcloud/discover/step.yaml'.
        Successfully loaded step data.
        Module 'tmt.steps.discover.shell' already imported.
        status: done
        Read file '/var/tmp/tmt/run-025/plans/testcloud/discover/tests.yaml'.
        Workdir '/var/tmp/tmt/run-025/plans/testcloud/provision' already exists.
        Read file '/var/tmp/tmt/run-025/plans/testcloud/provision/step.yaml'.
        Successfully loaded step data.
        Module 'tmt.steps.provision.testcloud' already imported.
        status: done
        Read file '/var/tmp/tmt/run-025/plans/testcloud/provision/guests.yaml'.
        Module 'tmt.steps.provision.testcloud' already imported.
        Workdir '/var/tmp/tmt/run-025/plans/testcloud/prepare' already exists.
        Read file '/var/tmp/tmt/run-025/plans/testcloud/prepare/step.yaml'.
        Successfully loaded step data.
        Module 'tmt.steps.prepare.shell' already imported.
        status: done
        Workdir '/var/tmp/tmt/run-025/plans/testcloud/execute' already exists.
        Read file '/var/tmp/tmt/run-025/plans/testcloud/execute/step.yaml'.
        Successfully loaded step data.
        Module 'tmt.steps.execute.internal' already imported.
        status: done
        Read file '/var/tmp/tmt/run-025/plans/testcloud/execute/results.yaml'.
        Workdir '/var/tmp/tmt/run-025/plans/testcloud/report' already exists.
        Read file '/var/tmp/tmt/run-025/plans/testcloud/report/step.yaml'.
        Successfully loaded step data.
        Module 'tmt.steps.report.display' already imported.
        status: done
        Workdir '/var/tmp/tmt/run-025/plans/testcloud/finish' already exists.
        Read file '/var/tmp/tmt/run-025/plans/testcloud/finish/step.yaml'.
        Successfully loaded step data.
        Module 'tmt.steps.finish.shell' already imported.
        status: todo
        Read file '/var/tmp/tmt/run-025/plans/testcloud/provision/step.yaml'.
        Successfully loaded step data.
        Module 'tmt.steps.provision.testcloud' already imported.
        Read file '/var/tmp/tmt/run-025/plans/testcloud/provision/guests.yaml'.
        Module 'tmt.steps.provision.testcloud' already imported.
        Step is done, not touching its data.
        Using the 'ProvisionTestcloud' plugin for the 'virtual' method.
            step is done, not overwriting plugin data
            Waking up testcloud instance 'tmt-025-stLYXxCU'.
            testcloud version: 0.9.10
        Provision wake up complete (already done before).
    Stopping guests in run '/var/tmp/tmt/run-025' plan '/plans/testcloud'.
        finish
        workdir: /var/tmp/tmt/run-025/plans/testcloud/finish

            Stopping testcloud instance 'tmt-025-stLYXxCU'.
            guest: stopped
            Removing testcloud instance 'tmt-025-stLYXxCU'.
            guest: removed
            Workdir '/var/tmp/tmt/run-025/plans/testcloud/provision/default-0' already exists.
            summary: 0 tasks completed
        status: done
        Write file '/var/tmp/tmt/run-025/plans/testcloud/finish/step.yaml'.

lnie@fedora:~/test-tmt$ virsh list --all
 Id   Name               State
 1    tmt-024-AexPXgUW   running

lnie@fedora:~/test-tmt$ tmt clean runs -vvdd --id run-025
clean
runs
    Removing workdir '/var/tmp/tmt/run-025'.
lnie@fedora:~/test-tmt$ rpm -q tmt
tmt-1.31.0-1.fc39.noarch

skycastlelily commented 8 months ago

For mrack created guest users will see:https://github.com/teemtee/tmt/issues/2796,but that's a different problem:)

skycastlelily commented 7 months ago

Hi, @martinpitt,would please check tmt clean guests --vvddl and then tmt clean runs -vvddl fix your problem?Thanks:) Btw, you need to start from the beginning,ie ,tmt run --until report provision --how virtual --image fedora-39

martinpitt commented 7 months ago

Starting from a clean slate: Fresh reboot, /var/tmp/tmt is empty, no running VMs.

tmt try -l -v fedora-rawhide
tmt clean guests -vvddl
tmt clean runs -vvddl

That works fine indeed (tmt-1.32.1-1.fc39.noarch). So cleaning up the last VM/run works, thanks for the tip!

But if I start a second session, then -l only ever cleans run-002, run-001 stays around even if I run the clean commands a second time.

--help and the command names suggest that they would clean all runs, though?

skycastlelily commented 7 months ago

So cleaning up the last VM/run works, thanks for the tip!

My pleasure.

But if I start a second session, then -l only ever cleans run-002, run-001 stays around even if I run the clean commands a second time.

That's because run-001 is NOT last run, run-001 is the second to last run:) I understand your request: when the last run is deleted, the second to last run becomes the last run,and so on. With current implement,tmt doesn't support that.

On Sun, Apr 7, 2024 at 3:29 PM Martin Pitt @.***> wrote:

Starting from a clean slate: Fresh reboot, /var/tmp/tmt is empty, no running VMs.

tmt try -l -v fedora-rawhide tmt clean guests -vvddl tmt clean runs -vvddl

That works fine indeed (tmt-1.32.1-1.fc39.noarch). So cleaning up the last VM/run works, thanks for the tip!

But if I start a second session, then -l only ever cleans run-002, run-001 stays around even if I run the clean commands a second time.

--help and the command names suggest that they would clean all runs, though?

— Reply to this email directly, view it on GitHub https://github.com/teemtee/tmt/issues/2772#issuecomment-2041349923, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKFR23BXUSFXM6TJNUG5MP3Y4DYVHAVCNFSM6AAAAABE62QY2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGM2DSOJSGM . You are receiving this because you commented.Message ID: @.***>

martinpitt commented 7 months ago

I understand your request: when the last run is deleted, the second to last run becomes the last run,and so on.

That'd be nice, but it's actually not my request. It was just an attempt to work around. The request here is that the clean command works without -l.

skycastlelily commented 7 months ago

tmt clean runs works well here:

@.:~/tmt/tmt$ ls /var/tmp/tmt/ instances run-001 run-002 testcloud testcloud.lock @.:~/tmt/tmt$ tmt clean runs -vvddl clean runs Workdir '/var/tmp/tmt/run-002' already exists. Removing workdir '/var/tmp/tmt/run-002'. @.:~/tmt/tmt$ tmt clean runs clean runs @.:~/tmt/tmt$ ls /var/tmp/tmt/ instances testcloud testcloud.lock @.***:~/tmt/tmt$ rpm -q tmt tmt-1.32.1-1.fc39.noarch

Would you please check whether there is a run.yaml file in your /var/tmp/tmt/run-001 dir?Thanks.

On Mon, Apr 8, 2024 at 4:37 PM Martin Pitt @.***> wrote:

I understand your request: when the last run is deleted, the second to last run becomes the last run,and so on.

That'd be nice, but it's actually not my request. It was just an attempt to work around. The request here is that the clean command works without -l.

— Reply to this email directly, view it on GitHub https://github.com/teemtee/tmt/issues/2772#issuecomment-2042180569, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKFR23FCVQTMXRLS2ZQGZOTY4JJNFAVCNFSM6AAAAABE62QY2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBSGE4DANJWHE . You are receiving this because you commented.Message ID: @.***>

martinpitt commented 7 months ago

Sorry for the delay! No, nothing in my /var/tmp/tmt/run-001/ leftover from a few days ago. I created a new one from tmt try, and that didn't create one either:

$ tmt try fedora-40
$ ls -d /var/tmp/tmt/run-00*
/var/tmp/tmt/run-001  /var/tmp/tmt/run-002
$ find /var/tmp/tmt/run-00* -name run.y*
$

I started a "real" test, and that does create one:

$ tmt run --until report provision --how virtual --image fedora-40
$ find /var/tmp/tmt/run-00* -name run.y*
/var/tmp/tmt/run-003/run.yaml

and tmt clean indeed cleans up run-003, but not the other two. So I suppose at least tmt try doesn't write these run.yaml files. I don't know any more what run-001/ did.

skycastlelily commented 7 months ago

Yes,that is because tmt try doesn't create a run.yaml file,and tmt clean //guests/runs commands without --id/--last will only clean things from the runs whose dir has a run.yaml file in it. That is a bug of tmt, however, you could workaround that by specify the --id of the run or use --last if what you want to clean are from the last run.

psss commented 3 months ago

@skycastlelily, do I understand correctly that this issue should now be fixed by #2889?

skycastlelily commented 3 months ago

Yep, but https://github.com/teemtee/tmt/pull/2882 and https://github.com/teemtee/tmt/pull/2805 are still valid for robustness reason,gonna to rebase that two merge requests:)

teemtee / tmt

tmt clean guests|runs ... doesn't #2772