search

crocovert / networks

GNU General Public License v3.0
19 stars 3 forks source link

qgis_process gives a segmentation fault with Networks enabled #15

Closed florisvdh closed 8 months ago

florisvdh commented 9 months ago

Some changes appear necessary to make the Networks plugin work with the command-line utility qgis_process (see its docs), which is installed with QGIS; see output below.

While qgis_process plugins enable networks is successful at making the plugin active (as also confirmed from the QGIS GUI), this command and subsequent qgis_process plugins and qgis_process list commands end up with a segmentation fault.

Running qgis_process plugins disable networks successfully makes it inactive after which the error is gone, but of course then Networks is inaccessible (including in QGIS GUI).

Potentially similar things have been solved in the past with Lat Lon Tools, QNeat3, Viewshed Analysis and Valhalla (linking here the related issues).

$ qgis_process plugins
Available plugins
(* indicates loaded plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
  networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
$ 
$ qgis_process plugins enable networks
Enabling plugin: "networks"
Enabled networks (Networks)

Available plugins
(* indicates enabled plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
* networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing

Segmentation fault (core dumped)
$ 
$ qgis_process plugins
Available plugins
(* indicates loaded plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
* networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
Segmentation fault (core dumped)
$ 
$ qgis_process list 1> /dev/null
double free or corruption (out)
Aborted (core dumped)
$
$ qgis_process plugins disable networks
Disabling plugin: "networks"
Disabled networks

Available plugins
(* indicates enabled plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
  networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
$ 
$ qgis_process plugins
Available plugins
(* indicates loaded plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
  networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
$ 
$ qgis_process --version
QGIS 3.32.3-Lima 'Lima' (67d46100b5)
QGIS code revision 67d46100b5
Qt version 5.15.3
Python version 3.10.12
GDAL/OGR version 3.6.4
PROJ version 9.1.1
EPSG Registry database version v10.076 (2022-08-31)
GEOS version 3.11.1-CAPI-1.17.1
SQLite version 3.37.2
OS Linux Mint 21.2

This was first observed in macOS, in an issue reported by a user of the R package {qgisprocess}, which calls qgis_process as backend to run QGIS processing algorithms in R.

crocovert commented 9 months ago

Hello,

With my version of QGis 3.32.3 and my version of Networks Plugin 2.7.11 on windows it's working. What versions are you using?

florisvdh commented 9 months ago

QGIS 3.32.3 on Linux Mint 21.2 (~ Ubuntu Jammy 22.04); see:

$ qgis_process --version
QGIS 3.32.3-Lima 'Lima' (67d46100b5)
QGIS code revision 67d46100b5
Qt version 5.15.3
Python version 3.10.12
GDAL/OGR version 3.6.4
PROJ version 9.1.1
EPSG Registry database version v10.076 (2022-08-31)
GEOS version 3.11.1-CAPI-1.17.1
SQLite version 3.37.2
OS Linux Mint 21.2

In https://github.com/r-spatial/qgisprocess/issues/179 @sriramab reported the segfault on macOS 12.6 with both QGIS 3.28.11 and 3.32.3:

QGIS 3.28.11-Firenze 'Firenze' (d1af706d752)
QGIS code revision d1af706d752
Qt version 5.15.2
Python version 3.9.5
GDAL/OGR version 3.3.2
PROJ version 8.1.1
EPSG Registry database version v10.028 (2021-07-07)
GEOS version 3.9.1-CAPI-1.14.2
SQLite version 3.35.2
OS macOS 12.6
QGIS 3.32.3-Lima 'Lima' (67d46100b5b)
QGIS code revision 67d46100b5b
Qt version 5.15.2
Python version 3.9.5
GDAL/OGR version 3.3.2
PROJ version 8.1.1
EPSG Registry database version v10.028 (2021-07-07)
GEOS version 3.9.1-CAPI-1.14.2
SQLite version 3.35.2
OS macOS 12.6
florisvdh commented 9 months ago

In my case Networks is at 2.7.11.

crocovert commented 9 months ago

Hello, The source of the error in this case was a 'mapCanvas' (below) which seems not relevant in my plugin code I've looked at the changes in the code made to fix the bug. I don't see a difference in my code that could explain this bug. I've followed adivices from Qgis documentation of qgis_process

I will try tomorrow with a linux desktop to see if the same issue appears

Best regards

$ qgis_process plugins Traceback (most recent call last): File "/usr/lib/python3/dist-packages/qgis/utils.py", line 423, in _startPlugin plugins[packageName] = package.classFactory(iface) File "/home/floris/.local/share/QGIS/QGIS3/profiles/default/python/plugins/latlontools/init.py", line 3, in classFactory return LatLonTools(iface) File "/home/floris/.local/share/QGIS/QGIS3/profiles/default/python/plugins/latlontools/latLonTools.py", line 27, in init self.canvas = iface.mapCanvas() AttributeError: 'NoneType' object has no attribute 'mapCanvas'

florisvdh commented 9 months ago

Thanks.

The solved issues in the other plugins are not necessarily the same as this one though; I've added them here as they might provide some further insight, but I'm not familiar with coding QGIS plugins at all. With those plugins the issue was not a segmentation fault, but rather no succes in detecting or loading the plugin at all.

IIRC the only other time I encountered a segfault in qgis_process was with the PCRaster provider plugin; the fix to solve this was not in the plugin though but in PCRaster directly (https://github.com/pcraster/pcraster/commit/5fc7b8c, https://github.com/pcraster/pcraster/issues/378).

crocovert commented 9 months ago

Hello,

I've tried with a PC ubuntu 22.04 with QGis 3.32.2. It works but I have a segmentation fault. I don't really know how to fix it

florisvdh commented 9 months ago

Yes, the segmentation fault is the problem we found too. I have no experience with developing QGIS Plugins I'm afraid. The segfault points at some inappropriate memory handling, either directly by the code or by a dependency that you use.

florisvdh commented 9 months ago

Some potential leads to debug: https://stackoverflow.com/a/58825725, https://stackoverflow.com/a/10035594, https://stackoverflow.com/q/2346806

crocovert commented 8 months ago

I've uploaded a new version which seems to fix the bug (at least on my ubuntu 23.04 dist). Tell me if it solves your issue

florisvdh commented 8 months ago

Thanks!! Yes, the segfault is solved :rocket: .

However another problem now pops up, which is also reported by QGIS when installing the plugin from zip-file. (BTW version 2.7.12 is not at plugins.qgis.org, maybe due to the same reason)

The problem is that a vitesse_velo script is imported but it is missing in the plugin. I presume this is the cause for inconsistent behaviour reported by qgis_process wrt the plugin being enabled or not.

Also the QGIS GUI throws an error when enabling the plugin.

afbeelding

$ /home/floris/git_repositories2/QGIS/build-6e3364f/output/bin/qgis_process plugins
Available plugins
(* indicates loaded plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
  networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
$ 
$ /home/floris/git_repositories2/QGIS/build-6e3364f/output/bin/qgis_process plugins enable networks
Enabling plugin: "networks"
Traceback (most recent call last):
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 423, in _startPlugin
    plugins[packageName] = package.classFactory(iface)
  File "/home/floris/.local/share/QGIS/QGIS3/profiles/default/python/plugins/networks/__init__.py", line 40, in classFactory
    from .networks_provider_plugin import NetworksPlugin
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/floris/.local/share/QGIS/QGIS3/profiles/default/python/plugins/networks/networks_provider_plugin.py", line 41, in <module>
    from .networks_provider import NetworksProvider
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/floris/.local/share/QGIS/QGIS3/profiles/default/python/plugins/networks/networks_provider.py", line 84, in <module>
    from .vitesse_velo import  vitesse_velo
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
ModuleNotFoundError: No module named 'networks.vitesse_velo'

Enabled networks (Networks)

Available plugins
(* indicates enabled plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
* networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
$ 
$ /home/floris/git_repositories2/QGIS/build-6e3364f/output/bin/qgis_process plugins
Traceback (most recent call last):
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 423, in _startPlugin
    plugins[packageName] = package.classFactory(iface)
  File "/home/floris/.local/share/QGIS/QGIS3/profiles/default/python/plugins/networks/__init__.py", line 40, in classFactory
    from .networks_provider_plugin import NetworksPlugin
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/floris/.local/share/QGIS/QGIS3/profiles/default/python/plugins/networks/networks_provider_plugin.py", line 41, in <module>
    from .networks_provider import NetworksProvider
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/floris/.local/share/QGIS/QGIS3/profiles/default/python/plugins/networks/networks_provider.py", line 84, in <module>
    from .vitesse_velo import  vitesse_velo
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
ModuleNotFoundError: No module named 'networks.vitesse_velo'

error starting plugin: networks

Available plugins
(* indicates loaded plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
  networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
$ 
$ /home/floris/git_repositories2/QGIS/build-6e3364f/output/bin/qgis_process plugins enable networks
Enabling plugin: "networks"
Plugin is already enabled!
$ 
$ /home/floris/git_repositories2/QGIS/build-6e3364f/output/bin/qgis_process plugins
Traceback (most recent call last):
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 423, in _startPlugin
    plugins[packageName] = package.classFactory(iface)
  File "/home/floris/.local/share/QGIS/QGIS3/profiles/default/python/plugins/networks/__init__.py", line 40, in classFactory
    from .networks_provider_plugin import NetworksPlugin
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/floris/.local/share/QGIS/QGIS3/profiles/default/python/plugins/networks/networks_provider_plugin.py", line 41, in <module>
    from .networks_provider import NetworksProvider
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/floris/.local/share/QGIS/QGIS3/profiles/default/python/plugins/networks/networks_provider.py", line 84, in <module>
    from .vitesse_velo import  vitesse_velo
  File "/home/floris/git_repositories2/QGIS/build-6e3364f/output/python/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
ModuleNotFoundError: No module named 'networks.vitesse_velo'

error starting plugin: networks

Available plugins
(* indicates loaded plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
  networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
$ 
$ /home/floris/git_repositories2/QGIS/build-6e3364f/output/bin/qgis_process plugins disable networks
Disabling plugin: "networks"
Disabled networks

Available plugins
(* indicates enabled plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
  networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
crocovert commented 8 months ago

Hello,

I had forgotten vitesse_velo.py in the github repository. Thanks Could you try please this new version, if it's working

Thanks

florisvdh commented 8 months ago

You solved it all, thanks a lot!

$ /home/floris/git_repositories2/QGIS/build-6e3364f/output/bin/qgis_process plugins
Available plugins
(* indicates loaded plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
  networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
$ 
$ /home/floris/git_repositories2/QGIS/build-6e3364f/output/bin/qgis_process plugins enable networks
Enabling plugin: "networks"
Enabled networks (Networks)

Available plugins
(* indicates enabled plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
* networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
$ 
$ /home/floris/git_repositories2/QGIS/build-6e3364f/output/bin/qgis_process plugins
Available plugins
(* indicates loaded plugins which implement Processing providers)

* QNEAT3
* valhalla
* processing_saga_nextgen
* networks
* pcraster_tools
* ViewshedAnalysis
* cartography_tools
* network_gt
* wbt_for_qgis
* latlontools
* grassprovider
* otbprovider
* processing
crocovert commented 8 months ago

Ok, thank you, I can close the issue

sriramab commented 8 months ago

@crocovert @florisvdh thank you for the efforts. Do you know how long it takes it takes to get into the qgis plugins repo for end user to install it ?