Feature Request: Ability to use existing GDAL installed via QGIS/OSGeo4W

[Saijin-Naib]

Blender and OS versions

Blender: v2.83.0 MS Store OS: Windows 10 x64 Pro v2004 20152.1000

Describe the Feature Request

I'd like the ability to use the existing GDAL install shipped with QGIS/OSGeo4W installs on Windows. Ideally, this would work automatically by pulling the proper path from the registry, but an option to set the GDAL path manually within BlenderGIS would also be very welcome.

Reason for Request

Compiling GDAL from source under Python on Windows is difficult if you don't have a pretty hefty Visual Studio development environment setup already. Alternatively, you must use something like CONDA or another Python stack that includes compiled GDAL, which precludes easily using the MS Store distributed Python environment.

[leonardofmed]

That would be really interesting if possible.

[domlysz]

Unfortunately it's not so easy because Blender ship it's own Python installation so in all cases it's mandatory to install the gdal-python module for it and this module is very sensitive to the version of Python and of GDAL : everything must perfectly match even the compiler used to produce GDAL binaries. It can make things very tricky, so I think it's harder than install the precompiled whl as proposed in the wiki (there is no compilation from source).

[Saijin-Naib]

@domlysz , this likely has nothing to do with BlenderGIS, but when one uses the Blender distributed via the MS Store, there is no way to add python wheels to the built-in Blender python environment, as the Blender app is within a limited-privilege sandbox that the user can not write to.

This use-case is why I proposed the above. It looks like I have to take this up with the Blender team directly. Do you know the best channel for this?

(And also, thank you SO much for BlenderGIS. I've seen some amazing work being done in the community with your tool, and I was hoping to give it a go).

[Saijin-Naib]

Can BlenderGIS recognize GDAL/Pillow/PyProj from other locations than the Blender python environment?

For instance, if I use the python.exe -m pip install --target=C:\Users\USERNAME\AppData\Roaming\Blender Foundation\Blender\2.83\scripts\addons\BlenderGIS-master GDAL.whl approach, Blender/BlenderGIS still doesn't recognize them

[domlysz]

Here some info on how Python search for package directories : https://leemendelowitz.github.io/blog/how-does-python-find-packages.html

Maybe it can works if both Python version perfectly match

[Saijin-Naib]

Is it possible for BlenderGIS to bundle PyProj/Pillow/GDAL?

So from your post, I take it BlenderGIS doesn't look anywhere for Python modules other than the python path, correct?

[Saijin-Naib]

From one of the Blender Windows Store maintainers on how to install modules into the Blender: from within store version of Blender;:

import subprocess as sp
import pathlib
import bpy

modpath = pathlib.Path(bpy.utils.script_path_user(), "modules")

sp.run([bpy.app.binary_path_python, "-m", "pip", "install", "--target", f"{modpath}", "pillow"], stdout=sb.PIPE)

After this restart Blender and import the module

Furthermore:

but you could create an operator that does these steps for you so, for your add-on first check if your dependencies exist. If not, add an operator that allows user to install the dependencies once done tell user to restart Blender, you should find the dependencies now

[jesterKing]

It could be possible to 'protect' the dependency loading - if they don't exist (haven't been installed yet) you could add an operator for the user to run that does this for them instead. The following works (tested with pillow only, but likely works for the rest as well) - note that it is just a script you could run directly in Blender, it is not directly operator code, but illustrates the solution:

import subprocess as sp
import pathlib
import bpy

# Installing to the %APPDATA%\Blender Foundation\Blender\2.83\scripts\modules path,
# which should be writable by the user. Tested to work in MS Store Blender version 2.83.1
modules_path = pathlib.Path(bpy.utils.script_path_user(), "modules")

sp.run([bpy.app.binary_path_python, "-m", "pip" , "install", "--target", f"{modules_path}", "pillow"], stdout=sp.PIPE)

Once this has been run the user needs to restart their Blender instance, after which an import PIL should work just fine.

FWIW, I've been responsible for releasing the MS Store version of Blender

[s-leger]

Be super carefull with too simple solutions to setup modules, it will not work everywhere. On some systems sites-package may be read only so you must setup in user's site-package. Also recent changes in blender api disabled access to system wide site packages (path not setup by default). You are also likely to break existing installations with package dependancy like setup a scipy will break blender's numpy with more recent version. So you must ensure that a writeable user site-package exists, and never install deps if they are present.

[domlysz]

Maybe I'm missing something but this code does not answer the original question about how to allows Blender own Python install to use system wide Python modules. It just run pip through the Python exe bundle with Blender and install the module inside a target addon folder instead of site-package one. There is nothing different than running pip directly from a terminal as suggested in wiki. But, I know nothing about Windows store so I don't really understand what the problem with it.

Also Stephen already proposed a way to install automatically some dependencies : https://github.com/domlysz/BlenderGIS/issues/221 However it does not solve the most tricky part : installing GDAL which is the most interesting module in our case as BlenderGIS does not need Pillow at all, and Pyproj functionalities are all covered by GDAL.

[jesterKing]

An option to allow Blender Python to use system environment variables like PYTHONPATH exists: --python-use-system-env but the problem is that giving command-line options to the store version of Blender will be a very involved process. I admit I have not tried this myself.

Using the method described above you'll have always knowledge of where you can install the dependencies. Naturally you should first figure out if you need to install dependencies, and only install those that are missing.

By installing to the location given by pathlib.Path(bpy.utils.script_path_user(), "modules") you don't have to worry about messing up your site-package, as it will be contained to the Blender installation only.