search

qgis / QGIS

QGIS is a free, open source, cross platform (lin/win/mac) geographical information system (GIS)
https://qgis.org
GNU General Public License v2.0
10.52k stars 2.99k forks source link

SAGA Upslope Area tool error: tool need graphical user interface #44514

Closed jvdkwast closed 1 year ago

jvdkwast commented 3 years ago

What is the bug or the crash?

The SAGA Upslope Area tool in the QGIS Processing Toolbox gives the following error: tool need graphical user interface [Upslope Area] and doesn't produce the output. image

Steps to reproduce the issue

  1. Click right on pixel in the DEM and copy the coordinate with the same CRS as the Filled DEM (306271.5,5622234.9 in the sample dataset).
  2. In the Processing toolbox go to SAGA | Terrain Analysis - Hydrology | Upslope Area
  3. Paste the X and Y coordinate copied from step 1.
  4. Choose the Filled DEM as Elevation, use Method = [0] Deterministic 8. Keep the rest as default.
  5. Click Run image

Filled_DEM_sample.zip

Versions

The tool worked well until QGIS 3.16.4 with the SAGA 2.x version. The error is observed in version 3.16.9 and 3.20.1 (I didn't test other vesions, but users report the same problem). The error is also observed with the SAGA Next Gen experimental plugin.

QGIS version | 3.16.9-Hannover | QGIS code revision | 9f8d2f79 -- | -- | -- | -- Compiled against Qt | 5.15.2 | Running against Qt | 5.15.2 Compiled against GDAL/OGR | 3.3.1 | Running against GDAL/OGR | 3.3.1 Compiled against GEOS | 3.9.1-CAPI-1.14.2 | Running against GEOS | 3.9.1-CAPI-1.14.2 Compiled against SQLite | 3.35.2 | Running against SQLite | 3.35.2 PostgreSQL Client Version | 13.0 | SpatiaLite Version | 5.0.1 QWT Version | 6.1.3 | QScintilla2 Version | 2.11.5 Compiled against PROJ | 8.1.0 | Running against PROJ | Rel. 8.1.0, July 1st, 2021 OS Version | Windows 10 Version 2009 Active python plugins | GeoCoding; quick_map_services; db_manager; MetaSearch; processing; processing_saga_nextgen

Additional context

In my opinion the LTR versions should use the stable SAGA processing tools. Now all SAGA tools give a warning and there's no (easy?) way to downgrade QGIS on Windows to version 3.16.4 when it was still working okay.

agiudiceandrea commented 3 years ago

there's no (easy?) way to downgrade QGIS on Windows to version 3.16.4

All the previously released QGIS versions for Windows are available at https://qgis.org/downloads/?C=M;O=D or https://download.osgeo.org/qgis/windows/?C=M&O=D as indicated on the QGIS Web site https://qgis.org/en/site/forusers/download.htm

The QGIS Long-term releases (3.16.9 included) in the old OSGeo4W repository uses the SAGA version 2.3.2, while the same QGIS Long-term releases in the OSGeo4W v2 repository uses SAGA version 7.8.2.

jvdkwast commented 3 years ago

True, but that's not easy for users who have first installed the new one and find out that it doesn't work and then have to use another installer, because you can't downgrade with the same installer. I made a youtube video today to explain it for my target group and point them to the installer: https://youtu.be/B1LMOOPMWgo.

It would be better for users if the LTR doesn't change these dependencies so they can longer rely on the functionality in 3.16.x in this case.

gioman commented 3 years ago

True, but that's not easy for users who have first installed the new one and find out that it doesn't work and then have to use another installer, because you can't downgrade with the same installer. I made a youtube video today to explain it for my target group and point them to the installer: https://youtu.be/B1LMOOPMWgo.

It would be better for users if the LTR doesn't change these dependencies so they can longer rely on the functionality in 3.16.x in this case.

@jvdkwast before we were stuck to SAGA 2. was the one shipped by default with several mainstream Linux distributions, in the meantime several new (better) SAGA versions were released. But at some point the same distributions started shipping SAGA 7. so finally we could move on... at some point it was necessary to do the jump.

Regarding this specific issue:

if this tool changed, meaning that on SAGA 2 didn't required SAGA gui and now it does, then I don't think there is much we can do as in QGIS SAGA tools that require SAGA gui are not supported. Well... at least this was true in the past, not 100% sure is still the case.

jvdkwast commented 3 years ago

@gioman Thanks for the explanation. If we're sure that the Upslope Area tool needs the SAGA GUI and this can't be fixed then the tool should be removed from the processing toolbox. Catchment delineation can then still been done in QGIS using workflows with GRASS, WhiteboxTools or PCRaster instead of SAGA.

gioman commented 3 years ago

Area tool needs the SAGA GUI and this can't be fixed then the tool should be removed from the processing toolbox

right, I will check if the tool cannot really run without saga gui.

agiudiceandrea commented 3 years ago

@gioman there are two tools in SAGA-GIS with the same name "Upslope Area": one (ta_hydrology 3) is interactive and cannot be run on CLI, and one (ta_hydrology 4) is not interactive and can be run on CLI via saga_cmd.

With the new (v2/msi) OSGeo4W repository installers, the "description" files for the SAGA tools are generated using SAGA itself (via saga_cmd dev_tools 7), while in the old (v1/exe) OSGeo4W repository the "description" files are taken from those maintained in the QGIS source code.

For the "Upslope Area" tool, the generated description file (ta_hydrology_upslope_area.txt in \apps\qgis-ltr\python\plugins\processing\algs\saga\description) is:

Upslope Area
ta_hydrology
QgsProcessingParameterRasterLayer|TARGET|Target Area|None|True
QgsProcessingParameterNumber|TARGET_PT_X|Target X coordinate|QgsProcessingParameterNumber.Double|0.000000|False|None|None
QgsProcessingParameterNumber|TARGET_PT_Y|Target Y coordinate|QgsProcessingParameterNumber.Double|0.000000|False|None|None
QgsProcessingParameterRasterLayer|ELEVATION|Elevation|None|False
QgsProcessingParameterRasterLayer|SINKROUTE|Sink Routes|None|True
QgsProcessingParameterRasterDestination|AREA|Upslope Area
QgsProcessingParameterEnum|METHOD|Method|[0] Deterministic 8;[1] Deterministic Infinity;[2] Multiple Flow Direction;[3] Multiple Triangular Flow Directon;[4] Multiple Maximum Downslope Gradient Based Flow Directon|False|2
QgsProcessingParameterNumber|CONVERGE|Convergence|QgsProcessingParameterNumber.Double|1.100000|False|0.001000|None

The issue is due to the fact that the first line should be Upslope Area|4 instead of Upslope Area in order to let saga_cmd execute the correct tool ("ta_hydrology 4" and not "ta_hydrology 3").

So the issue for the "Upslope Area" tool in the new (v2/msi) OSGeo4W repository is upstream in SAGA-GIS and not in QGIS.

@jvdkwast you can locally workaround the issue (in order to use the "Upslope Area" SAGA tool even with QGIS installed from the new v2/msi OSGeo4W repository) just substituting the first line of the file ta_hydrology_upslope_area.txt (in your_qgis_installation_folder\apps\qgis-ltr\python\plugins\processing\algs\saga\description) from Upslope Area to Upslope Area|4

gioman commented 3 years ago

@gioman there are two tools in SAGA-GIS with the same name "Upslope Area": one (ta_hydrology 3) is interactive and cannot be run on CLI, and one (ta_hydrology 4) is not interactive and can be run on CLI via saga_cmd.

@agiudiceandrea thanks for checking this.

With the new (v2/msi) OSGeo4W repository installers, the "description" files for the SAGA tools are generated using SAGA itself (via saga_cmd dev_tools 7), while in the old (v1/exe) OSGeo4W repository the "description" files are take from those maintained in the QGIS source code.

@jef-n using the description files generated by SAGA is really sub optimal (i.e. tools with parameters that are incompatible with each other, so we need to split the tool). We need the flexibility to split tools, tweak/fix description files, etc. Can we go back as we did in the past (if yes I can do the necessary PR with the description files we need for SAGA 7.*)? Also... does the above apply only to Windows/OSGeo4W or also Linux builds?

agiudiceandrea commented 3 years ago

@gioman issue reported to SAGA-GIS devs: https://sourceforge.net/p/saga-gis/mailman/message/37332776/ See also https://github.com/qgis/QGIS-Enhancement-Proposals/issues/230

jef-n commented 3 years ago

@jef-n using the description files generated by SAGA is really sub optimal (i.e. tools with parameters that are incompatible with each other, so we need to split the tool). We need the flexibility to split tools, tweak/fix description files, etc. Can we go back as we did in the past (if yes I can do the necessary PR with the description files we need for SAGA 7.*)? Also... does the above apply only to Windows/OSGeo4W or also Linux builds?

We neither have SAGA 2 nor 7 everywhere. OSGeo4W v2 has 7 and the old v1. The LTR is still available in both. 3.20 and master are however only available in v2 - so updating 3.20 and above should be fine.

agiudiceandrea commented 3 years ago

@gioman @jef-n issue supposedly fixed in the generated interface description for SAGA 8.0.0 https://sourceforge.net/p/saga-gis/mailman/message/37354740/

gioman commented 3 years ago

@gioman @jef-n issue supposedly fixed in the generated interface description for SAGA 8.0.0 https://sourceforge.net/p/saga-gis/mailman/message/37354740/

@agiudiceandrea do you confirm they have no GIT repo where we could do some PRs with fixes to their description files?

alexbruy commented 3 years ago

@gioman they do have a GIT repository, but there are no description files. They are generated by the special tool. So if you want to fix something you have to fix that utility. But as far as I can see, there is no easy way to add very specific fixes like splitting tool into several.

agiudiceandrea commented 3 years ago

@gioman the mirror of the SAGA-GIS repository on GitHub is at https://github.com/saga-gis/saga-gis It seems to me the devs are kindly open to fix the tool that generates the description files. I've reported various issues in the past using the SAGA-GIS mailing list at https://sourceforge.net/p/saga-gis/mailman/ and they have been fixed in some days or weeks.

gioman commented 3 years ago

@gioman the mirror of the SAGA-GIS repository on GitHub is at https://github.com/saga-gis/saga-gis It seems to me the devs are kindly open to fix the tool that generates the description files. I've reported various issues in the past using the SAGA-GIS mailing list at https://sourceforge.net/p/saga-gis/mailman/ and they have been fixed in some days or weeks.

@agiudiceandrea as @alexbruy said we need specific things like splitting tools that have parameters that are not compatible with each other. This make me more convinced we need to take over of that description files management, as we always did.

alexbruy commented 3 years ago

Probably it is better to concentrate on the improving one of the existing 3rd party plugins then?

gioman commented 3 years ago

Probably it is better to concentrate on the improving one of the existing 3rd party plugins then?

@alexbruy yes, I agree. I have it in my todo list since long.

jcm36 commented 2 years ago

Hi, is there any update on this? This is the main issue preventing me from being able to update. I have a few custom QGIS models created that use the SAGA Upslope Area tool.

Thanks,

agiudiceandrea commented 2 years ago

@jcm36 as written in a previous comment, to fix the issue you just need to substitute the first line of the file ta_hydrology_upslope_area.txt (in your_qgis_installation_folder\apps\qgis-ltr\python\plugins\processing\algs\saga\description) from Upslope Area to Upslope Area|4

jcm36 commented 2 years ago

Ok thanks. Will try at work tomorrow. Could that fix be made in upcoming versions?

Ludwig-K commented 2 years ago

On QGis 3.22/Windows the file ta_hydrology_upslope_area.txt is under [qgis_install_dir]\apps\qgis\python\plugins\sagaprovider\description The workaround runs like a charm...

AdrianoEnge commented 2 years ago

After I change the first line of the file ta_hydrology_upslope_area.txt (in your_qgis_installation_folder\apps\qgis-ltr\python\plugins\processing\algs\saga\description) from Upslope Area to Upslope Area|4. The problem still happens in my project

DusanEastcode commented 2 years ago

Did anyone try to fix this on mac?

agiudiceandrea commented 2 years ago

@DusanEastcode, maybe somebody could help you if you specify some basic info about your system and your issue, e.g.: the complete error messages text, the exact QGIS version installed, the installer used to install QGIS, the SAGA-GIS version installed, the SAGA processing plugin installed and used, the mac OS version. Anyway, have you tried to search for the ta_hydrology_upslope_area.txt (or something like this) description file in your QGIS installation folder?

DusanEastcode commented 2 years ago

Anyway, have you tried to search for the ta_hydrology_upslope_area.txt (or something like this) description file in your QGIS installation folder?

@agiudiceandrea This file is not available on MacOS. There are only dylib files. Anyway I tried to install virtual machine with Win x64bit but editing given .txt file didn't help me.

agiudiceandrea commented 2 years ago

@DusanEastcode, could you please specify the requested basic info about your systems? The folder containing the SAGA-GIS "description" files is located at the path returned by the SagaUtils.sagaDescriptionPath() if the default sagaprovider core plugin is used:

import os
from sagaprovider import SagaUtils
print(os.path.abspath(SagaUtils.sagaDescriptionPath()))
DusanEastcode commented 2 years ago

@agiudiceandrea thanks for your response. I will try to find it. When I use this util I get: 


Traceback (most recent call last):
  File "/Applications/QGIS.app/Contents/MacOS/../Resources/python/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "<string>", line 2, in <module>
  File "/Applications/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 799, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
ModuleNotFoundError: No module named 'sagaprovider'```
DusanEastcode commented 2 years ago

@agiudiceandrea I downloaded QGIS-LTR 3.22 and now I see these files. Unfortunately the content of the UpslopeArea.txt reference the correct dependency as you can see in first row. The 

Upslope Area|4
ta_hydrology
QgsProcessingParameterRasterLayer|TARGET|Target Area|None|True
QgsProcessingParameterNumber|TARGET_PT_X|Target X coordinate|QgsProcessingParameterNumber.Double|0.0|False|None|None
QgsProcessingParameterNumber|TARGET_PT_Y|Target Y coordinate|QgsProcessingParameterNumber.Double|0.0|False|None|None
QgsProcessingParameterRasterLayer|ELEVATION|Elevation|None|False
QgsProcessingParameterRasterLayer|SINKROUTE|Sink Routes|None|True
QgsProcessingParameterEnum|METHOD|Method|[0] Deterministic 8;[1] Deterministic Infinity;[2] Multiple Flow Direction
QgsProcessingParameterNumber|CONVERGE|Convergence|QgsProcessingParameterNumber.Double|1.1|False|None|None
QgsProcessingParameterRasterDestination|AREA|Upslope Area

While running upslope are I got different error :) (see attached log) qgis.log

DusanEastcode commented 2 years ago

@agiudiceandrea
now in 3.22-LTR whitetower I again get this when trying to do upslope area from coordinates 

ta_hydrology "Upslope Area"  -TARGET_PT_X 5531667.91 -TARGET_PT_Y 1765540.3 -ELEVATION "/private/var/folders/ql/4964x3_92px1yd_fxdnh2lyh0000gn/T/processing_MhgbZb/b8a80c4b0f894a8989b1fdea030f3ad3/filledClippedSinks.sgrd" -METHOD 0 -CONVERGE 1.1 -AREA "/clipped5LTR/upslope.sdat"Processing(0): SAGA execution console output
____________________________

#####   ##   #####    ##
###     ###  ##       ###
###   # ## ##  #### # ##
### ##### ##    # #####
##### #   ##  ##### #   ##
____________________________

SAGA Version: 7.3.0

Error: tool needs graphical user interface [Upslope Area]Python warning(1): warning:<input>:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=b'/var/folders/ql/4964x3_92px1yd_fxdnh2lyh0000gn/T/tmpjwecwj0j.py' mode='r' encoding='UTF-8'>

@agiudiceandrea does this work for you on mac in latest stable LTR? I mean 3.22

hydrogeology990 commented 2 years ago

After I change the first line of the file ta_hydrology_upslope_area.txt (in your_qgis_installation_folder\apps\qgis-ltr\python\plugins\processing\algs\saga\description) from Upslope Area to Upslope Area|4. The problem still happens in my project

This is happening to me too. First I tried replacing Upslope Area with Upslope Area|4, but that didn't work. Also, that folder path is different in my version (QGIS version - 3.22.8-Białowieża). Instead it's saved here: C:\Program Files\QGIS 3.22.8\apps\qgis-ltr\python\plugins\sagaprovider\description\ta_hydrology_upslope_area.txt Then I tried to install the "Processing Saga NextGen Provider" plug in, which got rid of the message saying SAGA 7.8.2 is not officially supported, but it still didn't fix the problem.

Additionally, I noticed the ta_hydrology_upslope_area file is different too:

Upslope Area|4 ta_hydrology QgsProcessingParameterRasterLayer|TARGET|Target Area|None|True QgsProcessingParameterNumber|TARGET_PT_X|Target X coordinate|QgsProcessingParameterNumber.Double|0.000000|False|None|None QgsProcessingParameterNumber|TARGET_PT_Y|Target Y coordinate|QgsProcessingParameterNumber.Double|0.000000|False|None|None QgsProcessingParameterRasterLayer|ELEVATION|Elevation|None|False QgsProcessingParameterRasterLayer|SINKROUTE|Sink Routes|None|True QgsProcessingParameterRasterDestination|AREA|Upslope Area QgsProcessingParameterEnum|METHOD|Method|[0] Deterministic 8;[1] Deterministic Infinity;[2] Multiple Flow Direction;[3] Multiple Triangular Flow Directon;[4] Multiple Maximum Downslope Gradient Based Flow Directon|False|2 QgsProcessingParameterNumber|CONVERGE|Convergence|QgsProcessingParameterNumber.Double|1.100000|False|0.001000|None

Error log for reference: error log.txt

hydrogeology990 commented 2 years ago

It seems to be working now in version 3.26.1 after a few changes:

  1. Replacing the text, 'Upslope Area' with 'Upslope Area|4' in the ta_hydrology_upslope_area.txt file.
  2. Opening the Project menu, then the 'Properties popup menu and re-selecting the correct 'coordinate reference system'
  3. Changing the assigned coordinate reference system (under Source) of the DEM file used in the 'Elevation' input to match the the same coordinate reference system, and then using the 'Coordinate capture' plug-in to 'Start capture' before re-entering the new co-ordinates into the Upslope Area tool.
alexbruy commented 1 year ago

SAGA Provider was not maintained and now remowed from QGIS (see #50834). Users are encouraged to use the 3rd-party plugins instead.

DusanEastcode commented 1 year ago

SAGA Provider was not maintained and now remowed from QGIS (see #50834). Users are encouraged to use the 3rd-party plugins instead.

Can you please navigate community which plugin to use? I am not able too google any 👀

nicogodet commented 1 year ago

See https://github.com/qgis/QGIS-Enhancement-Proposals/issues/230

DusanEastcode commented 1 year ago

See qgis/QGIS-Enhancement-Proposals#230

Thanks I am using SAGA Next Gen plugin and still get this error :) But I see there are others so I am gonna to try them. Is there someone who successfully delineated watershed meanwhile? :)

nicogodet commented 1 year ago

@DusanEastcode already reported https://github.com/north-road/qgis-processing-saga-nextgen/issues/16 but not fixed yet

DusanEastcode commented 1 year ago

@DusanEastcode already reported north-road/qgis-processing-saga-nextgen#16 but not fixed yet

so since April no response :)

wanderzen91 commented 1 year ago

@DusanEastcode after editing the ta_hydrology_upslope_area.txt you shall use the processing.run('saga:upslopearea', alg_params) and not the processing.run('sagang:upslopearea', alg_params) :)

DusanEastcode commented 1 year ago

@DusanEastcode after editing the ta_hydrology_upslope_area.txt you shall use the processing.run('saga:upslopearea', alg_params) and not the processing.run('sagang:upslopearea', alg_params) :)

but I am running it on mac ;)