Fix pip installation issues when blender is installed system wide

[Mateusz-Grzelinski]

Install dependencies in known, user owned location instead of random python interpreter (usually owned by blender).

• Current behaviour: installs to .\blender-4.2.0-windows-x64\4.2\python\Lib\site-packages
• New behaviour: installs to %appdata%\Blender Foundation\Blender\4.2\scripts\modules (path indicated by bpy.utils.user_resource("SCRIPTS", path="modules")).
• Additionally pin dependency for compability with python 3.7 (requests<=2.29.0)

Reference: https://docs.blender.org/manual/en/4.2/editors/preferences/file_paths.html#script-directories

Pros:

• does not break existing setups
• This should fix any error related to PermissionError: [Errno 13] (windows) for system wide installations of blender.
• this can potentially solve multiple issues:

  • 173 #169 #105 #99 (probably) #94 #93 #73 (already solved) #70 #37 (maybe) #31

• pip install ensurepip has --upgrade flag specified what might periodically cause permission denied error

This kind of error is usually fixed by manually running (or similar):

.\blender-4.2.0-windows-x64\4.2\python\bin\python.exe -m pip install flask --target .\blender-4.2.0-windows-x64\4.2\python\Lib\site-packages

This change does not break existing user setup: the pip install is called only when "test import" fails. It is also fine to have some packages in old place and some packages in new dir

Cons:

• is it breaking change? TODO Specify example. I did not find any breaking corner cases.

TODOs:

• [x] verify blender 2.8 compability (waits for #174) - ok
• [x] verify 4.2 compability - ok
• [x] verify the change on bigger project and try to find issues - tested on ImagePaste and other smaller addons
• [x] verify load order from sys.path does not mess with debugging - seems fine
• [x] ensure if reinstalling packages does not throw an error (executing pip install command again). This is very unlikely, but I want to be sure. - ok
• [x] depracate blender.allowModifyExternalPython
• [x] old blender installation (2.80) start to have problem with newest dependencies -
  • pin requests to 2.29.0 to fix blender 2.80

Different design options:

• try to install in blender python system dir (site-packages), if fails, fallback to installation in modules
  • a bit more code, but doable. I do not like this idea as it messes with blender default instalation and the modules are very tricky to uninstall.
• Make the required dependencies self contained with VS code extension
  • chellange: blender ships with different python versions.
• it can be setting or interactive option: Where do you want to install python dependencies?

Squash merge after 172 - no particular reason, lets take it one by one.

[Mateusz-Grzelinski]

For documentation purposes: Reinstaling dependencies in blender 2.80 ends with error (I forced this very unlikely scenario by removing all checks):

Execute:  E:\BlenderProjects\bledner.exe\blender-2.80-windows64\blender-2.80-windows64\2.80\python\bin\python.EXE -m pip install requests --target C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules
Collecting requests
  Using cached https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
  Using cached https://files.pythonhosted.org/packages/1c/d5/c84e1a17bf61d4df64ca866a1c9a913874b4e9bdc131ec689a0ad013fb36/certifi-2024.7.4-py3-none-any.whl
Collecting idna<4,>=2.5 (from requests)
  Using cached https://files.pythonhosted.org/packages/e5/3e/741d8c82801c347547f8a2a06aa57dbb1992be9e948df2ea0eda2c8b79e8/idna-3.7-py3-none-any.whl
Collecting charset-normalizer<4,>=2 (from requests)
  Using cached https://files.pythonhosted.org/packages/96/fc/0cae31c0f150cd1205a2a208079de865f69a8fd052a98856c40c99e36b3c/charset_normalizer-3.3.2-cp37-cp37m-win_amd64.whl
Collecting urllib3<3,>=1.21.1 (from requests)
  Using cached https://files.pythonhosted.org/packages/d2/b2/b157855192a68541a91ba7b2bbcb91f1b4faa51f8bae38d8005c034be524/urllib3-2.0.7-py3-none-any.whl
Installing collected packages: certifi, idna, charset-normalizer, urllib3, requests
Successfully installed certifi-2024.7.4 charset-normalizer-3.3.2 idna-3.7 requests-2.31.0 urllib3-2.0.7
Target directory C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\certifi already exists. Specify --upgrade to force replacement.
Target directory C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\certifi-2024.7.4.dist-info already exists. Specify --upgrade to force replacement.
Target directory C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\charset_normalizer already exists. Specify --upgrade to force replacement.
Target directory C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\charset_normalizer-3.3.2.dist-info already exists. Specify --upgrade to force replacement.
Target directory C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\idna already exists. Specify --upgrade to force replacement.
Target directory C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\idna-3.7.dist-info already exists. Specify --upgrade to force replacement.
Target directory C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\requests already exists. Specify --upgrade to force replacement.
Target directory C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\requests-2.31.0.dist-info already exists. Specify --upgrade to force replacement.
Target directory C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\urllib3 already exists. Specify --upgrade to force replacement.
Target directory C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\urllib3-2.0.7.dist-info already exists. Specify --upgrade to force replacement.
Target directory C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\bin already exists. Specify --upgrade to force replacement.
You are using pip version 10.0.1, however version 24.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
Traceback (most recent call last):
  File "e:\BlenderProjects\blender_vscode\pythonFiles\launch.py", line 28, in <module>
    addons_to_load=addons_to_load,
  File "e:\BlenderProjects\blender_vscode\pythonFiles\include\blender_vscode\__init__.py", line 21, in startup
    installation.ensure_packages_are_installed(["debugpy", "flask", "requests"])
  File "e:\BlenderProjects\blender_vscode\pythonFiles\include\blender_vscode\installation.py", line 20, in ensure_packages_are_installed
    install_packages(package_names)
  File "e:\BlenderProjects\blender_vscode\pythonFiles\include\blender_vscode\installation.py", line 32, in install_packages
    ensure_package_is_installed(name)
  File "e:\BlenderProjects\blender_vscode\pythonFiles\include\blender_vscode\installation.py", line 39, in ensure_package_is_installed
    install_package(name)
  File "e:\BlenderProjects\blender_vscode\pythonFiles\include\blender_vscode\installation.py", line 48, in install_package
    if not module_can_be_imported(name):
  File "e:\BlenderProjects\blender_vscode\pythonFiles\include\blender_vscode\installation.py", line 76, in module_can_be_imported
    __import__(name)
  File "C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\requests\__init__.py", line 43, in <module>
    import urllib3
  File "C:\Users\grzel\AppData\Roaming\Blender Foundation\Blender\2.80\scripts\modules\urllib3\__init__.py", line 42, in <module>
    "urllib3 v2.0 only supports OpenSSL 1.1.1+, currently "
ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'OpenSSL 1.1.0h  27 Mar 2018'. See: https://github.com/urllib3/urllib3/issues/2168

After some reading it seems that oldee python version will start to have problem with openssl dependencies.

[JacquesLucke]

Deprecating allowModifyExternalPython seems ok if we are not accidentally modifying the global Python installation.

[Mateusz-Grzelinski]

All work is done, if no comments, I will merge on weekend.

[Mateusz-Grzelinski]

tested on ubuntu 22.04.4, with apt installed blender 3.0.1 - works fine