CLI `--verbosity` flag behaviour depends on position

[eimrek]

Describe the bug

During some testing with @giovannipizzi we found that the --verbosity flag in verdi commands depends on the position.

Very simple demo is

(aiida) kristjan@ubuntu:~$ verdi --verbosity error help

where the output is empty.

Whereas

(aiida) kristjan@ubuntu:~$ verdi help --verbosity error
Warning: You are currently using a post release development version of AiiDA: 2.5.1.post0
Warning: Be aware that this is not recommended for production and is not officially supported.
Warning: Databases used with this version may not be compatible with future releases of AiiDA
Warning: as you might not be able to automatically migrate your data.

outputs the Warning about the aiida version, as shown.

Expected behavior

Intuitively, both of the examples above should probably hide the Warning.

Or does the --verbosity flag work in a 'hieractical' manner, such that when writing it after the verdi subcommand (instead of directly after verdi), it only applies to some subset of the logging?

Your environment

• Operating system: ubuntu
• Python version: 3.9
• aiida-core version: main

[sphuber]

Or does the --verbosity flag work in a 'hieractical' manner, such that when writing it after the verdi subcommand (instead of directly after verdi), it only applies to some subset of the logging?

It is not intended to work like that in any case. I think the problem here is that the warning you see is emitted when the profile is loaded. This is done by the -p/--profile option, which is only defined on the verdi base command. I think that click processes callbacks for commands sequentially, going from top level commands down through the subcommands. So when you pass the -v flag for verdi, that is processed first, properly setting the loglevel and then the -p option is processed, at which point the warning emitted by loading the profile is ignored. However, if you pass -v for the subcommand, the -p is processed before the logging.

I am not sure how we can improve this. I see two main approaches:

1. Change/control the order in command/option callback processing order of click.
2. Remove the -v/--verbosity option from all subcommands

The first one seems very tricky because the parsing order is tightly controlled by click. The second would be easy, as we are currently dynamically adding the verbosity flag to all subcommands. But this would hurt the UX, since we intentionally did this as it is convenient to be able to pass verbosity for all subcommands and not be forced to pass it directly after verdi and before any subcommands.

In the end, I think this inconsistency only happens with log messages that are emitted during the loading of the profile. Given that I don't see a real alternative to fix this, I think that is acceptable. But if anyone sees a way to improve this, that would be great of course.