Robinlovelace
commented
1 year ago Full context and work-around provided in the commit above.
Closed Robinlovelace closed 1 year ago
Robinlovelace
commented
1 year ago Full context and work-around provided in the commit above.
florisvdh
commented
1 year ago Hi @Robinlovelace thanks for the issue. Can you paste the output of qgis_version(debug = TRUE) and cat(qgis_result_stdout(dem_wetness))? I suspect this is an upstream issue, not R's. Maybe the sagang provider or SAGA / QGIS itself. Considering that I get no problem here to get and access the .tif files, it may be due to used versions of the software.
library(qgisprocess)
#> Attempting to load the cache ... Success!
#> QGIS version: 3.30.1-'s-Hertogenbosch
#> Having access to 1289 algorithms from 9 QGIS processing providers.
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
#> >>> Run `qgis_enable_plugins()` to enable 1 disabled plugin(s) and
#> access their algorithms: cartography_tools
dem <- system.file("raster/dem.tif", package = "spDataLarge")
dem_wetness <- qgis_run_algorithm("sagang:sagawetnessindex",
DEM = dem)
#> Argument `WEIGHT` is unspecified (using QGIS default value).
#> Using `AREA = qgis_tmp_raster()`
#> Using `SLOPE = qgis_tmp_raster()`
#> Using `AREA_MOD = qgis_tmp_raster()`
#> Using `TWI = qgis_tmp_raster()`
#> Argument `SUCTION` is unspecified (using QGIS default value).
#> Using `AREA_TYPE = "[0] total catchment area"`
#> Using `SLOPE_TYPE = "[0] local slope"`
#> Argument `SLOPE_MIN` is unspecified (using QGIS default value).
#> Argument `SLOPE_OFF` is unspecified (using QGIS default value).
#> Argument `SLOPE_WEIGHT` is unspecified (using QGIS default value).
dem_wetness_twi <-
qgis_extract_output(dem_wetness, "TWI") |>
qgis_as_terra()
dem_wetness_twi
#> class : SpatRaster
#> dimensions : 117, 117, 1 (nrow, ncol, nlyr)
#> resolution : 30.85, 30.85 (x, y)
#> extent : 794599.1, 798208.6, 8931775, 8935384 (xmin, xmax, ymin, ymax)
#> coord. ref. : WGS 84 / UTM zone 17S (EPSG:32717)
#> source : filed01c5b288977.tif
#> name : filed01c5b288977
#> min value : 7.214644
#> max value : 18.347366Created on 2023-04-26 with reprex v2.0.2
This is using the qgis_extract_output() function from the single_output_fvdh branch.
Robinlovelace
commented
1 year ago You should be able to reproduce the issue with...
docker run -it geocompr/geocompr:qgis /bin/bashOr similar. Will check now...
Robinlovelace
commented
1 year ago I get:
qgis_version(debug = TRUE)
[1] "3.30.1-'s-Hertogenbosch"
Versions reported by 'qgis_process':
------------------------------------
QGIS 3.30.1-'s-Hertogenbosch ''s-Hertogenbosch' (447e8d717a)
QGIS code revision 447e8d717a
Qt version 5.15.3
Python version 3.10.6
GDAL/OGR version 3.4.1
PROJ version 8.2.1
EPSG Registry database version v10.041 (2021-12-03)
GEOS version 3.10.2-CAPI-1.16.0
SQLite version 3.37.2
OS Ubuntu 22.04.2 LTS
I can still reproduce the issue. Agree, it may be due to upstream changes associated with that particular version of QGIS. Do we need a more recent version? @jannes-m was advocating for that but I switched to the default versions of stuff due to an issue (which is fixable) in rocker images.
jannes-m
commented
1 year ago I think this problem disappeared when using the ubuntugis unstable installation (with Ubuntu 22.04) though the qgis version was the same. In any case I wouldn't bet that it stays this way or is this way on every other platform. This problem already exists for too long...
Robinlovelace
commented
1 year ago Good point Jannes. If it can be a self-fixing issue I'm happy : )
Worth fixing upstream? I've created this placeholder, v. happy if you or anyone can have a bash: https://github.com/geocompx/docker/issues/37 and that may fix it :crossed_fingers:
florisvdh
commented
1 year ago I think this problem disappeared when using the ubuntugis unstable installation
Yes that may be it. The Ubuntu repo has SAGA 7.3, the ubuntugis-unstable PPA has 8.2.2.
$ apt policy saga
saga:
Installed: 8.2.2+dfsg-1~jammy0
Candidate: 8.2.2+dfsg-1~jammy0
Version table:
*** 8.2.2+dfsg-1~jammy0 500
500 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu jammy/main amd64 Packages
100 /var/lib/dpkg/status
7.3.0+dfsg-7 500
500 http://ftp.belnet.be/ubuntu jammy/universe amd64 PackagesThe output of cat(qgis_result_stdout(dem_wetness)) will show you which SAGA version was called.
Robinlovelace
commented
1 year ago That's good context, fixed in later versions of SAGA, and not worth supporting ephemeral out-of-date versions. Happy for this to be closed, think we need to the issue our side in the qgis image we're shipping.
florisvdh
commented
1 year ago Yes, better to not create workarounds in the R package to manage upstream problems, unless they are widespread or not getting solved. Better is to post the issue upstream if not already fixed.
For continuous integration purposes in combination with the ubuntugis-unstable PPA (= where the official GRASS release lives), you should probably use QGIS from its ubuntugis-nightly repo, since that repo provides instant QGIS rebuilds against the ubuntugis-unstable PPA. That means, whenever GRASS etc. is updated at the ubuntugis-unstable PPA, the regular ubuntugis QGIS build (which too is tailored for using that PPA) remains out-of-sync with the PPA (until start of next month) and may temporarily not be installable together with GRASS from the PPA (this situation occurred a month ago in this repo).
Feel free to close when you think this needs no further effort here. Discussion can continue of course.
Robinlovelace
commented
1 year ago Thanks @florisvdh agreed, this is not an issue for this codebase. Ported to https://github.com/geocompx/docker/issues/37. Good discussion and many thanks for the pointers to well supported repos.
florisvdh
commented
1 year ago Assuming that the conflict has to do with the SAGA version, and that older SAGA versions only supported its native .sdat output format (by design), it means following approaches seem possible when trying to meet (older) SAGA expectations – the last one only in theory though:
gsub() trick):
qgis_tmp_*() generated filepaths, and explicitly define the future output filepaths (ending with .sdat) as extra arguments to qgis_run_algorithm("sagang:sagawetnessindex", DEM = dem, AREA =, TWI =, .....).options(qgisprocess.tmp_raster_ext, ".sdat") before calling qgis_run_algorithm(). That should work (not tested). And perhaps set back to previous state after the processing, so that the default ".tif" extension of qgis_tmp_raster() is back in place.qgis_sanitize_arguments() (soon an internal function) needs to be aware of the SAGA version context when it sets a missing rasterDestination-type argument as qgis_tmp_raster(). To do that through qgis_process, it would need to call some dummy sagang algorithm first just to be able to parse some stdout and get hold of the SAGA version, so qgis_run_algorithm() would need to dispatch such situation. Clearly that would be too awkward and specific if the problem is only marginal.
xiaoxuan6722
commented
4 months ago library(qgisprocess) Attempting to load the package cache ... Success! QGIS version: 3.36.0-Maidenhead Having access to 698 algorithms from 7 QGIS processing providers. Run
qgis_configure(use_cached_data = TRUE)to reload cache and get more details.qgis_search_algorithms("buffer")
A tibble: 11 × 5
provider provider_title group algorithm algorithm_title
1 gdal GDAL Vector geoprocessing gdal:buffervectors Buffer vectors 2 gdal GDAL Vector geoprocessing gdal:onesidebuffer One side buffer 3 grass GRASS Raster (r.*) grass:r.buffer r.buffer 4 grass GRASS Raster (r.*) grass:r.buffer.lowmem r.buffer.lowmem 5 grass GRASS Vector (v.*) grass:v.buffer v.buffer 6 native QGIS (native c++) Vector geometry native:buffer Buffer 7 native QGIS (native c++) Vector geometry native:bufferbym Variable width b… 8 native QGIS (native c++) Vector geometry native:multiringconstantbuffer Multi-ring buffe… 9 native QGIS (native c++) Vector geometry native:singlesidedbuffer Single sided buf… 10 native QGIS (native c++) Vector geometry native:taperedbuffer Tapered buffers 11 native QGIS (native c++) Vector geometry native:wedgebuffers Create wedge buf… qgis_search_algorithms("wetness") # A tibble: 0 × 5 # ℹ 5 variables: provider , provider_title , group , algorithm , # algorithm_title topo_total=qgisprocess::qgis_run_algorithm( + alg="sagang:sagawetnessindex", + DEM=doubs_dem_utm_masked, + SLOPE_TYPE=1, + SLOPE= tempfile(fileext = ".sdat"), + AREA= tempfile(fileext = ".sdat"), + .quiet=TRUE + ) Error in `assert_qgis_algorithm()`: ! Can't find QGIS algorithm 'sagang:sagawetnessindex'. Run `qgis_algorithms()` for a list of available algorithms. Run `rlang::last_trace()` to see where the error occurred.
How can I solve this problem
xiaoxuan6722
commented
4 months ago library(qgisprocess) Attempting to load the package cache ... Success! QGIS version: 3.36.0-Maidenhead Having access to 698 algorithms from 7 QGIS processing providers. Run
qgis_configure(use_cached_data = TRUE)to reload cache and get more details.qgis_search_algorithms("buffer")
A tibble: 11 × 5
provider provider_title group algorithm algorithm_title
1 gdal GDAL Vector geoprocessing gdal:buffervectors Buffer vectors 2 gdal GDAL Vector geoprocessing gdal:onesidebuffer One side buffer 3 grass GRASS Raster (r.*) grass:r.buffer r.buffer 4 grass GRASS Raster (r.*) grass:r.buffer.lowmem r.buffer.lowmem 5 grass GRASS Vector (v.*) grass:v.buffer v.buffer 6 native QGIS (native c++) Vector geometry native:buffer Buffer 7 native QGIS (native c++) Vector geometry native:bufferbym Variable width b… 8 native QGIS (native c++) Vector geometry native:multiringconstantbuffer Multi-ring buffe… 9 native QGIS (native c++) Vector geometry native:singlesidedbuffer Single sided buf… 10 native QGIS (native c++) Vector geometry native:taperedbuffer Tapered buffers 11 native QGIS (native c++) Vector geometry native:wedgebuffers Create wedge buf… qgis_search_algorithms("wetness") # A tibble: 0 × 5 # ℹ 5 variables: provider , provider_title , group , algorithm , # algorithm_title topo_total=qgisprocess::qgis_run_algorithm( + alg="sagang:sagawetnessindex", + DEM=doubs_dem_utm_masked, + SLOPE_TYPE=1, + SLOPE= tempfile(fileext = ".sdat"), + AREA= tempfile(fileext = ".sdat"), + .quiet=TRUE + ) Error in `assert_qgis_algorithm()`: ! Can't find QGIS algorithm 'sagang:sagawetnessindex'. Run `qgis_algorithms()` for a list of available algorithms. Run `rlang::last_trace()` to see where the error occurred.
How can I solve this problem
Partial reprex:
"sagang:sagawetnessindex"returns not one but four rasters -- catchment area, catchment slope, modified catchment area, and topographic wetness index.We can read a selected output by providing an output name in the
qgis_as_terra()function.For full reproducible example, clone the
geocompx/geocomprrepo, checkout theqgisbranch and open in the supplied devcontainer or on a system that has the relevant dependencies installed.