search

teemtee / tmt

Test Management Tool
MIT License
84 stars 125 forks source link

Verbose mode not working in `tmt * show` commands #2364

Open psss opened 1 year ago

psss commented 1 year ago

It seems that a regression has been introduced in ec6ba65e which breaks tmt test show --verbose:

tmt test show smoke -v

/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag container and virtual
                    tier 0

Verbose mode should include sources, fmf-id and web link:

/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag container and virtual
                    tier 0
                 sources /home/psss/git/tmt/tests/main.fmf
                         /home/psss/git/tmt/tests/core/main.fmf
                         /home/psss/git/tmt/tests/core/smoke/main.fmf
                  fmf-id url: https://github.com/teemtee/tmt.git
                         ref: HEAD
                         name: /tests/core/smoke
                     web https://github.com/teemtee/tmt/tree/HEAD/tests/core/smoke/main.fmf

@happz, could you possibly have a look into this?

happz commented 1 year ago

@psss are you sure it's caused by the patch you mentioned? I'm getting the same outcome with 1.27.0:

[pts-2:0]: happz@multivac [(HEAD detached at 1.27.0)] ~/git/tmt $ type tmt
tmt is hashed (/usr/bin/tmt)
[pts-2:0]: happz@multivac [(HEAD detached at 1.27.0)] ~/git/tmt $ tmt --version
tmt version: 1.27.0 (9c35269a)
[pts-2:0]: happz@multivac [(HEAD detached at 1.27.0)] ~/git/tmt $ tmt test show smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0

[pts-2:0]: happz@multivac [(HEAD detached at 1.27.0)] ~/git/tmt $ tmt test show -v smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0

[pts-2:0]: happz@multivac [(HEAD detached at 1.27.0)] ~/git/tmt $ tmt test show -vv smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0

[pts-2:0]: happz@multivac [(HEAD detached at 1.27.0)] ~/git/tmt $

The position of -v seems to be the decisive factor:

[pts-2:0]: happz@multivac [(HEAD detached at 1.27.0)] ~/git/tmt $ tmt test show smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0

[pts-2:0]: happz@multivac [(HEAD detached at 1.27.0)] ~/git/tmt $ tmt -v test show smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0
                 sources /home/happz/git/tmt/tests/main.fmf
                         /home/happz/git/tmt/tests/core/main.fmf
                         /home/happz/git/tmt/tests/core/smoke/main.fmf
                  fmf-id url: github:teemtee/tmt
                         ref: HEAD
                         name: /tests/core/smoke
                     web github:teemtee/tmt/tree/HEAD/tests/core/smoke/main.fmf

[pts-2:0]: happz@multivac [(HEAD detached at 1.27.0)] ~/git/tmt $

The current main seems to be consistent with this:

[pts-2:0]: happz@multivac [main] ~/git/tmt $ hatch run dev:tmt test show smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0

[pts-2:0]: happz@multivac [main] ~/git/tmt $ hatch run dev:tmt -v test show smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0
                 sources /home/happz/git/tmt/tests/main.fmf
                         /home/happz/git/tmt/tests/core/main.fmf
                         /home/happz/git/tmt/tests/core/smoke/main.fmf
                  fmf-id url: github:teemtee/tmt
                         ref: main
                         name: /tests/core/smoke
                     web github:teemtee/tmt/tree/main/tests/core/smoke/main.fmf

[pts-2:0]: happz@multivac [main] ~/git/tmt $

I guess if there's a regression, it happened earlier than in ec6ba65eab805351369801bb54490a54fcc65cf4 :/ Might have happened when I was messing with loggers, for example.

It feels like tests and its subcommands ignore verbosity options:

[pts-2:0]: happz@multivac [main] ~/git/tmt $ hatch run dev:tmt test show -v smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0

[pts-2:0]: happz@multivac [main] ~/git/tmt $ hatch run dev:tmt test -v show smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0

[pts-2:0]: happz@multivac [main] ~/git/tmt $ hatch run dev:tmt -v test show smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0
                 sources /home/happz/git/tmt/tests/main.fmf
                         /home/happz/git/tmt/tests/core/main.fmf
                         /home/happz/git/tmt/tests/core/smoke/main.fmf
                  fmf-id url: github:teemtee/tmt
                         ref: main
                         name: /tests/core/smoke
                     web github:teemtee/tmt/tree/main/tests/core/smoke/main.fmf

[pts-2:0]: happz@multivac [main] ~/git/tmt $

And indeed, with this patch:

diff --git a/tmt/cli.py b/tmt/cli.py
index 059ae919..3b546710 100644
--- a/tmt/cli.py
+++ b/tmt/cli.py
@@ -531,7 +531,11 @@ def tests_show(context: Context, **kwargs: Any) -> None:
     Use '.' to select tests under the current working directory.
     """
     tmt.Test.store_cli_invocation(context)
-    for test in context.obj.tree.tests():
+
+    logger = context.obj.logger.clone() \
+        .apply_verbosity_options(**kwargs)
+
+    for test in context.obj.tree.tests(logger=logger):
         test.show()
         echo()

... I get what you're looking for:

[pts-2:0]: happz@multivac [main] ~/git/tmt $ hatch run dev:tmt test show smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0

[pts-2:0]: happz@multivac [main] ~/git/tmt $ hatch run dev:tmt test show -v smoke
/tests/core/smoke
                 summary Just a basic smoke test
                 contact Petr Šplíchal <psplicha@redhat.com>
                    test tmt --help
                    path /tests/core/smoke
               framework shell
                  manual false
                 require tmt
                duration 5m
                 enabled true
                  result respect
                     tag 'container' and 'virtual'
                    tier 0
                 sources /home/happz/git/tmt/tests/main.fmf
                         /home/happz/git/tmt/tests/core/main.fmf
                         /home/happz/git/tmt/tests/core/smoke/main.fmf
                  fmf-id url: github:teemtee/tmt
                         ref: main
                         name: /tests/core/smoke
                     web github:teemtee/tmt/tree/main/tests/core/smoke/main.fmf

[pts-2:0]: happz@multivac [main] ~/git/tmt $
happz commented 1 year ago

So, it's probably been around for some time already. What shall I do about it? :) I can propose a patch, but it won't be a one-liner.

psss commented 1 year ago

Yes, indeed, the ec6ba65e commit went to 1.25 so it's with us for quite some time (we definitely need to extend the test coverage here), was surprised myself that quick checking out old branches did not help. Btw, just learned that the commit page on GitHub gives a nice overview of tags where the commit is present:

obrazek

What shall I do about it? :) I can propose a patch, but it won't be a one-liner.

If you could propose a patch, that would be great. Whatever number of lines it would cost :)

psss commented 1 year ago

@happz, do you plan to work on this for 1.28 or shall we move it?

happz commented 1 year ago

@psss 1.29, please

happz commented 1 year ago

Moving to 1.30. I do have a draft, but I'm not happy about the shape it takes, and I want to find a better way of involving verbosity options in tests & co.

happz commented 11 months ago

I can't get the decorator right, trying to add pyright into the mix to help me with annotations in https://github.com/teemtee/tmt/pull/2520.