Conda path - Githubissues

joaoponceleao commented 8 months ago

Hi. Thanks for the new installer. I can see pip3 in the Contents/bin folder, but I can't see where the conda env is to activate it. How does one install new packages for plugins? I assume conda would be preferred over pip here (I realise plugins will call pip anyway). On a related note, please consider pinning any relevant conda packages to prevent dependency issues when installing or updating extra plugin dependencies.

m-kuhn commented 8 months ago

There is an activate script in Contents/lib/python3.11/venv/scripts/common/activate, does that help somehow? There is a conda executable packaged and used during the installation, however it is no longer on the system after the setup completes, this would need to be investigated. I did not yet look into how to manage a conda environment and more into how to get an installer for users that are not at ease with the command line. I would try to use pip first and let plugins do their work. Or install conda the command line way.

If we want a better integration of conda for plugin dependencies, I think we should do something like https://github.com/qgis/QGIS-Enhancement-Proposals/issues/202 or https://github.com/opengisch/qpip, where plugins can add a requirements.txt and we could ship a dedicated conda backend to install these.

joaoponceleao commented 8 months ago

Thanks for the input. Personally it would be great if we had access to conda here. My system conda install is not seeing the QGIS venv. I assume I can activate it with conda activate <path to qgis conda venv> and proceed with installs this way? A means to use conda to install packages would be beneficial, particularly with more ML tools and workflows relying on modules that cannot be easily installed through pip (pytorch and the like).

(having said that, I was quite surprised to see most of my dependencies install without issues with the QGIS pip here)

As for plugin dependencies, I agree that this should be up to plugin publishers who will simply use pip, but they too sometimes have to either provide instructions for custom installs, or produce hackish scripts for some edge cases (for instance, we added behaviour in our plugin to install miniconda in a custom folder just to add 'triangle' which tends to trip pip).

joaoponceleao commented 8 months ago

An update on this. Installing QGIS again removes all pip installs from the system, meaning users that have installed dependencies, as well as plugins, will have to reinstall them. I can imagine this becoming a problem for some plugins that have their requirement install behaviour inside some installation function rather than on load.

Could be worth exploring installing the venv somewhere outside the QGIS.app package (Lib/App Support or ~/.config maybe?), and add a menu item or an option on the installer to delete that location for those who need a clean install.

m-kuhn commented 8 months ago

I think ideally the venv would go into the profile folder (i.e. you can have one venv per profile, just like plugins), just like done with qpip listed above

joaoponceleao commented 8 months ago

That is a great idea. The only issue I can see is what happens when you change profiles. QGIS will pick up the new folder for packages, but will it also activate the new venv (with all the new c library paths)?

m-kuhn commented 8 months ago

C libraries are always a tricky topic, however, I think it boils down to this:

if a library is already installed on the base system (typical example gdal), ideally its corresponding python libs are installed on the base system as well
if a python libs for a base system library is installed in a virtualenv, only a wrapper can be installed. Loading the same C lib (in the same or different version) results in UB -> trouble ahead.
if a python lib with no corresponding C lib is installed, it's free to install the C lib also in the virtualenv. If two python installed modules do this approach we are in UB again.