Release as "stable"

[m-kuhn]

After the latest update, we were wondering if this plugin is ready for a stable release.

To get some confidence: @angusmcb @axelande as you have been recently testing it, what plugins did you test with and what platforms did you test on?

[axelande]

I've been testing it with Windows installed with the Network installer with the GeoDataFarm plugin

[angusmcb]

Hi, I was testing on Ubuntu. I have been, and still am, away, so haven't tested the latest update.

There may have been an issue when packages have sub dependencies, pip ignoring what was already installed on the system, so ending with two conflicting versions.

Won't be able to test properly for another two weeks.

[axelande]

I tried to run the plugin on an old Ubuntu machine and it "worked". I had some issues since the current version on plugins.qgis.org does not work at all on Ubuntu. However the updated version here in the repo "seems to work". However, it seems like it only works after installation and reopening QGIS. Having a QGIS requirement on qpip, installing that and then installing my plugin did not work (then qpip did not look into the plugin folder for a requirement before the installation).

[axelande]

I've continued to test the plugin and it is a little bit "hard to predict" what will happen when the new qpip is released, there are a lot of changes compared to the released version on plugins.qgis.org. I have some trouble "activating it" when I try, without the released version this seems to be really hard to test :p But it could be an issue that you need to restart QGIS before it works but it would be good to have a new version of qpip to test with.

[m-kuhn]

I've just released the latest code as v0.0.2

[angusmcb]

There seems to be a bug with the version numbering, so the latest version is not the one used by the plugin manager.

[m-kuhn]

Thanks, I just unapproved the beta versions, it should be good now. I guess it's a limitation of QGIS-Django.

[axelande]

I got the updated version, but it still the "same issue" (on both Ubuntu and Windows) that there is a restart of QGIS that is needed before the qpip plugin works as expected :(

I had a look at the QPIP logs:

2024-08-27T11:25:05     INFO    Adding C:\Users\AxelHor\AppData\Roaming\QGIS\QGIS3\profiles\default\python\dependencies to PYTHONPATH
2024-08-27T11:25:05     INFO    Adding C:\Users\AxelHor\AppData\Roaming\QGIS\QGIS3\profiles\default\python\dependencies\bin to PATH
2024-08-27T11:25:05     INFO    Applying monkey patch to qgis.utils

And the Python Error:

2024-08-27T11:25:07     CRITICAL    Traceback (most recent call last):
              File "C:\OSGeo4W/apps/qgis/./python\qgis\utils.py", line 426, in _startPlugin
              plugins[packageName] = package.classFactory(iface)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "C:\Users/AxelHor/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\geodatafarm\__init__.py", line 38, in classFactory
              from .GeoDataFarm import GeoDataFarm
              File "C:\OSGeo4W/apps/qgis/./python\qgis\utils.py", line 923, in _import
              mod = _builtin_import(name, globals, locals, fromlist, level)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "C:\Users/AxelHor/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\geodatafarm\GeoDataFarm.py", line 71, in 
              from .support_scripts.generate_reports import RapportGen
              File "C:\OSGeo4W/apps/qgis/./python\qgis\utils.py", line 923, in _import
              mod = _builtin_import(name, globals, locals, fromlist, level)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "C:\Users/AxelHor/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\geodatafarm\support_scripts\generate_reports.py", line 1, in 
              from reportlab.lib.pagesizes import letter, A4
              File "C:\OSGeo4W/apps/qgis/./python\qgis\utils.py", line 923, in _import
              mod = _builtin_import(name, globals, locals, fromlist, level)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
             ModuleNotFoundError: No module named 'reportlab'

One thing that strikes me is that qpip is loaded first and the log from the plugin occurred 2s before I got the "ModuleNotFoundError". I tried to install my plugin again but nothing from QPIP, no messages in the log module etc.

Before QGIS gets restarted there is no other messages in the QPIP message log. After restarting QGIS it works as expected.

I don't know how to debug this further or suggest a change :(

[axelande]

The problem after installation (if not restarted of QGIS) is that QGIS never gets reloaded after qgis.utils.loadPlugin = self.patched_load_plugin. If you e.g. use the python console and import qgis it will have the "correct" (from qpip) loadPlugin however, the interface itself have the original. If we can't reload qgis after this mokeypatch the user needs to get a warning that he/she needs to restart QGIS manually and try to install the other plugin again.

[m-kuhn]

If hot reloading doesn't eork, such a warning would indeed be better!

[m-kuhn]

@axelande @angusmcb I just released a new version 0.0.3, do you think this is ready for a stable release?

[axelande]

It worked great on one of my computers and failed on another: This is the traceback: pyplugin_installer.instance().onManagerClose()

Traceback (most recent call last): File "", line 1, in File "C:\Users/axa/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\a00_qpip\plugin.py", line 127, in patched_load_plugin dialog, run_gui = self.check_deps(additional_plugins=[packageName]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users/axa/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\a00_qpip\plugin.py", line 156, in check_deps if os.path.dirname(dist._path) != self.site_packages_path: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 277, in dirname File "", line 241, in split TypeError: expected str, bytes or os.PathLike object, not Path

I diged a little bit deeper and added this line log(f'dist: {dist._path}') to which dist that failed: I don't get why it failed, but maybe through a try expect block around that check?

2024-09-25T08:01:17 INFO dist: C:\Users\axa\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\qfieldsync\libqfieldsync.whl/libqfieldsync-1.0.dist-info/

(the update seems to work nicely apart from this minor error!)

[angusmcb]

At the moment PIP will install all dependencies of packages that it is installing. This is of course a useful feature, but leads to duplication if some of the dependencies are already installed (e.g. a package relying on numpy, matplotlib etc. that are probably already installed in the environment). How about only installing the specific dependencies specified in the plugin requirements.txt, without sub-dependencies (not a perfect solution....).

[m-kuhn]

I think there are two things here:

1. It would be good to have a blocklist of modules, especially binary ones (gdal, pyqt, ...), it is a known problem that installing these may crash QGIS and this plugin is at the right spot in the stack to fix this. It would be great to be able to filter the list of modules for these and warn/prevent.
2. Reinstalling already installed modules will be required in some instances (e.g. minimum version larger than already installed). Also installing transitive dependencies will be required in many cases. IIRC using --prefix instead of --target alongside pip install fixes at least some of these problems. Possibly someone could do some tests with that.

The question is, is one of these blocking a stable release or shall these be addressed down the road in dedicated tickets (or one dedicated ticket), I think they'll need a few experiments and potentially weigh competing interests to be addressed properly.

[axelande]

I don't think that those points block it, I think it would be nice to have this as a stable version and to be able to use it in other plugins and then more issues will be known etc. (have you had the possibility to look at the bug that I found caused by the qfieldsync plugin? and if the a try: except block solved that issue?)

[angusmcb]

Agreed that it works within the limits that it has. Can't wait until there are no bugs before releasing it!

[m-kuhn]

Agreed that it works within the limits that it has. Can't wait until there are no bugs before releasing it!

Is there something besides https://github.com/opengisch/qpip/issues/19 ?