Execution Environments don't support overriding `typeCheckingMode` option.

[GlebMorgan]

I'm using Pylance in VSCode. I want to specify different type checking settings / rules (at least type checking mode) for the application and tests separately.

Consider a simple project structure with app/ and tests/ directories:

project/
├── app/
│   ╰── main.py
├── tests/
│   ╰── test.py
╰── pyrightconfig.json

Say, I want type checking in test files to be more forgiving (since they are just tests). So, I decided to set typeCheckingMode to strict for files inside app/ and to standard – inside tests/. I suppose the intended way to do it is using execution environments (even though this reply states this is not possible at all).

Docs for "Execution Environment Options" say:

In addition, any of the type check diagnostics settings listed above can be specified. These settings act as overrides for the files in this execution environment.

The corresponding link leads to the section where typeCheckingMode option is also mentioned.

Great, let's try this:

pyrightconfig.json

{
"include": [ "app", "tests" ],
"executionEnvironments": [
{
"root": "app",
"typeCheckingMode": "strict"
},
{
"root": "tests",
"extraPaths": [ "app" ],
"typeCheckingMode": "basic"  // for example
}
]
}

Apparently, that does not work. Pylance continues to use the default value for typeCheckingMode option (standard) and ignores execution environment overrides.

Does it mean typeCheckingMode option is not overridable by execution environment settings and only rule overrides are supposed to work? Is that intended? Because for me these seem to always work in conjunction: typeCheckingMode specifies a general "base" profile (rule set) to start with and then specific rules can be further adjusted to be more or less strict.

OK, there seems to be a different approach – a strict option:

pyrightconfig.json

{
"include": [ "app", "tests" ],
"typeCheckingMode": "standard",
"strict": [ "app" ]
}

This kinda works at first glance, but in this case all rule overrides stop working for app/ directory. Neither global report... settings nor their execution environment counterparts have any effect on the files inside app/. Besides, if typeCheckingMode override would've worked for execution environments, strict option would seem kinda redundant in general...

There are some other alternatives:

• Don't use strict option, set typeCheckingMode option to strict globally and then specify a bunch of rule overrides for tests/ to make it essentially "basic" – this will quickly make the setting file a mess and kinda defeat the whole purpose of typeCheckingMode option as a baseline rule set.
• Use # pyright: standard in every test file as suggested here – better, but this approach kinda splits configuration between pyrightconfig.json and every test file (and also rule exceptions and type checking modes will be set in different places).
• Use a separate configuration file inside tests/ directory and link it with extends option – as extreme as it is, I still tried this approach but never got it to work. Besides, I believe extends is not even supposed to override stuff only for a subdirectory it is located in.
• Disable type checking for tests/ completely – that will work as a complete fallback option, but I would still like Pylance to give me some insights for test files, just without going too pedantic.

So, my question is: what is the intended way to have different Pylance/Pyright settings for different parts of the project? And why executionEnvironments option does not support typeCheckingMode override despite (as far as I understood) it should according to docs?

[erictraut]

Yes, you're correct that typeCheckingMode isn't supported for execution environments. That's very much by design. I don't think it would make sense to add this because typeCheckingMode implies a set of default severities for each of the type checking rules, but execution environments represent overrides to severities.

As you've noted, if you want to apply strict mode to a subset of directories or files, there is a strict setting.

[GlebMorgan]

OK, got it regarding typeCheckingMode overrides, thank you for an explanation 🙂

But then why strict option does not allow for any rule overrides on top of itself? As far as I experimented, once some files fall under strict scope, there is no way to adjust the rules applied to these files. The rule set for them is cemented to whatever strict mode uses by default.

[erictraut]

But then why strict option does not allow for any rule overrides on top of itself?

Here's the override order:

1. A typeCheckingMode setting in the main part of the config file establishes the defaults rule set
2. A specific rule setting in the main part of the config file (e.g. reportUnnecessaryCompare) overrides 1
3. A specific rule setting in an execution environment overrides previous
4. The strict setting overrides previous and applies the strict rule set
5. File-specific # pyright comments overrides previous
6. File-specific # pyright: ignore comments overrides previous
7. The ignore setting overrides previous

So, if you want to apply rules on top of the strict option, you can do so using file-specific # pyright comment.

The strict setting has a high precedence is based on my own experience managing large teams with large code bases. I want to ensure that certain directories (especially newly-written code) used strict-mode settings. I didn't want developers on my team to intentionally or unintentionally disable some of the strict-mode checks for those files in the config file. If a strict-mode check must be disabled for some reason within a specific file, we use a # pyright comment in that file and accompany it with a lengthy comment justifying why it's OK to disable that check for that file.

[GlebMorgan]

I very much appreciate your detailed answers 🙂

In my personal opinion, the whole purpose of more specific settings is to override more generic ones, otherwise generic options might not end up being useful at all, like in my example.

Ideal option IMHO would be to let each execution environment have its own ruleset (maybe even defined in a separate config seciton) and then deal with any exceptions via # pyright comments individually on per-file or per-line basis. But as of now, at least in usecases similar to mine the only way to find strict setting useful is to fully conform to the rule set it provides, since it is rigidly determined by Pylint. I would say that is not always favorable.

But I respect your experience, probably just looking at the functionality Pylance provides from a different perspective.