After the system python version update to 3.9, I get "No valid Python interpreter found" [PANTS 1.30. IDEA 2020.2/2020.3]

[slvrtrn]

Started to have this issue after Ubuntu 20.04 -> 21.04 upgrade. Verified in IDEA 2020.2 with the stable plugin and with 2020.3 bleeding-edge version (released on 24th of March 2021, Pants Support 1.17.0.7c71f505173a6499b6e0c91651e10ed03c6d2ff0)

PANTS work fine from the console (I am able to bootstrap the project and run the app or the tests), but it is not possible to refresh (or import) the project in IDEA.

I am using pyenv for the 3.8.6 setup. The actual package python3.8 is removed from Ubuntu 21.04 and replaced with python3.9 by default.

I also have .python-version file in the project root

 % cat .python-version 
3.8.6

if I run just python from the console in the project root, it starts the correct interpreter

 % python
Python 3.8.6 (default, May  4 2021, 09:31:45) 
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

that's also true for IDEA terminal - it runs 3.8.6. But PANTS plugin complains about the python version.

2021-05-06 11:26:22,750 [  46140]   INFO - pants.model.SimpleExportResult - Exit code 1 
2021-05-06 11:26:22,750 [  46140]   INFO - pants.model.SimpleExportResult - No valid Python interpreter found. For `pants_version = "1.30.0"`, Pants requires Python 3.6, 3.7, or 3.8 to run. Please check that a valid interpreter is installed and on your $PATH.

However,

~ % pyenv global
3.8.6

[slvrtrn]

I created a minimal repo to illustrate the issue: https://github.com/slvrtrn/pants-python-3.9-issue

and the log says

2021-05-06 11:26:22,750 [  46140]   INFO - pants.model.SimpleExportResult - Exit code 1 
2021-05-06 11:26:22,750 [  46140]   INFO - pants.model.SimpleExportResult - �[31mNo valid Python interpreter found. For `pants_version = "1.30.0"`, Pants requires Python 3.6, 3.7, or 3.8 to run. Please check that a valid interpreter is installed and on your $PATH.�[0m

in the actual terminal, everything works fine

serge@abyss ~/work/pants-python-3.9-issue
% ./pants compile :: 

<<<snip>>>

New virtual environment successfully created at /home/serge/.cache/pants/setup/bootstrap-Linux-x86_64/1.30.0_py38.
09:51:36 [INFO] initializing pantsd...
09:51:36 [INFO] Attempting to fetch watchman binary from: https://binaries.pantsbuild.org/bin/watchman/linux/x86_64/4.9.0-pants1/watchman ...
100% .................................................. 34581 KB 2.755s
09:51:39 [INFO] Fetched watchman binary from: https://binaries.pantsbuild.org/bin/watchman/linux/x86_64/4.9.0-pants1/watchman .
09:51:39 [INFO] pantsd initialized.

11:51:35 00:00 [main]
               (To run a reporting server: ./pants server)
11:51:39 00:04   [setup]
11:51:39 00:04     [parse]
               Executing tasks in goals: bootstrap -> imports -> unpack-jars -> unpack-wheels -> deferred-sources -> native-compile -> gen -> jvm-platform-validate -> resolve -> compile
11:51:39 00:04   [bootstrap]
11:51:39 00:04     [substitute-aliased-targets]
11:51:39 00:04     [jar-dependency-management]
11:51:39 00:04     [bootstrap-jvm-tools]
11:51:39 00:04     [provide-tools-jar]
11:51:39 00:04   [imports]
11:51:39 00:04     [ivy-imports]
11:51:39 00:04   [unpack-jars]
11:51:39 00:04     [unpack-jars]
11:51:39 00:04   [unpack-wheels]
11:51:39 00:04     [unpack-wheels]
11:51:39 00:04   [deferred-sources]
11:51:39 00:04     [deferred-sources]
11:51:39 00:04   [native-compile]
11:51:39 00:04     [conan-prep]
11:51:39 00:04     [conan-fetch]
11:51:39 00:04     [c-for-ctypes]
11:51:39 00:04     [cpp-for-ctypes]
11:51:39 00:04   [gen]
11:51:39 00:04     [protoc]
11:51:39 00:04     [thrift-java]
11:51:39 00:04     [thrift-py]
11:51:39 00:04     [py-thrift-namespace-clash-check]
11:51:39 00:04     [grpcio-prep]
11:51:39 00:04     [grpcio-run]
11:51:39 00:04   [jvm-platform-validate]
11:51:39 00:04     [jvm-platform-validate]
11:51:39 00:04   [resolve]

11:51:41 00:06       [coursier]
11:51:44 00:09       [coursier]
11:51:45 00:10   [compile]
11:51:45 00:10     [compile-jvm-prep-command]
11:51:45 00:10       [jvm_prep_command]
11:51:45 00:10     [compile-prep-command]
11:51:45 00:10     [compile]
11:51:45 00:10     [rsc]
11:51:45 00:10       [cache] 
                   No cached artifacts for 1 target.
                   Invalidated 1 target.
11:51:45 00:10       [isolation-mixed-pool-bootstrap] 
                   [1/1] Compiling 1 mixed source in 1 target (src/scala/com/github/slvrtrn:hello).

<<<snip>>>

[slvrtrn]

So, this seems to be a duplicate of https://github.com/pantsbuild/intellij-pants-plugin/issues/505

If I run the IDEA like this

% ~/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/202.8194.7/bin/idea.sh &

and NOT from the Toolbox, the correct version (3.8.6 in my case) is picked up by IDEA

If I run it from Toolbox, the default Python version is used (3.9.1) despite me having pyenv global 3.8.6 In other words, Pyenv is ignored completely.