Allow more cmd-line control over logging

[EvanKirshenbaum]

Currently, you can set the logging level by means of a --log-level argument, which takes a single level (e.g., info, debug, error). For finer control, you can specify a --log-config argument that points to a file that gives you almost complete control, but the syntax for that file is complicated and error-prone, to put it mildly.

In setup_logging(), there are two logging formats that are defined, a simple one, which is used when the logging level is INFO or above, and a detailed one (including thread name, file name, and line number) that's used when the level is DEBUG or below. A couple of libraries, whose loggers have DEBUG messages are set to be INFO if the specified level is below, in order to suppress them.

While working on #82, I ran into two problems:

• The aiohttp package logs an INFO message for every inbound call, which got quite annoying, so I added it to pin at ERROR (although I'm not sure that's right).
• There are debugging traces I'd like to add (that users don't need to see), but I really don't need all of the detailed information.

What I think I'd like to see is

1. Two or three named formats, perhaps compact for the one we currently use for INFO and detailed for the one we use for DEBUG,
2. The --log-level
   1. specified as [pkg:]level[:fmt], so you can say --log-level debug:compact or --log-level aiohttp:error, and
   2. able to be specified multiple times
3. Every logger in an imported package (I think that means everything with a . in its name) having a default level of, probably, WARN.
   • I'm not positive that this can be done, but it's worth a try.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jun 07, 2022 at 11:56 AM PDT. Closed on Jun 08, 2022 at 2:42 PM PDT.

[EvanKirshenbaum]

This issue was referenced by the following commit before migration:

• c61324ca7bd299c7cc83f28e269c9d66356dc436 by @EvanKirshenbaum on Jun 07, 2022 at 1:29 PM PDT

[EvanKirshenbaum]

It all appears to work, and it's essentially backward compatible. From the help string:

  --log-level SPEC      Configure logging. The format is <level>,
                        <name>:<level> or <level>:<format>, where <level> is
                        an integer or one of DEBUG, INFO, WARN, WARNING,
                        ERROR, and CRITICAL and <format> is one of COMPACT and
                        DETAILED. If <name> is specified, the given level is
                        used for that logger (or any children). If <name> is
                        'modules', the level is used (by default) for loggers
                        from all known imported modules. <level> and <format>
                        are case-insensitive. This argument may be specified
                        multiple times.

This is a bit of a fib, as you can actually specify both the name and the format, but if you do, the format is ignored. (That is, the format for the no-name level is used.)

The default level is INFO, and the default format is COMPACT at INFO and above and DETAILED below. We can easily add new formats just by adding to a dictionary.

The current list of packages used by modules specifications is matplotlib, PIL, and aiohttp. Unfortunately, I couldn't find any easy way to say "apply this to any existing or future loggers whose names have a dot in them".

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jun 08, 2022 at 2:41 PM PDT.