Closed Robinlovelace closed 3 years ago
The if I allow it to use the default Python install with
RETICULATE_PYTHON=/usr/bin/python
I get
library(sf)
#> Linking to GEOS 3.8.0, GDAL 3.0.2, PROJ 6.3.0
library(raster)
#> Loading required package: sp
library(RQGIS)
#> Loading required package: reticulate
set_env(dev = FALSE)
#> Assuming that your root path is '/usr'!
#> $root
#> [1] "/usr"
#>
#> $qgis_prefix_path
#> [1] "/usr/bin/qgis"
#>
#> $python_plugins
#> [1] "/usr/share/qgis/python/plugins"
data("incongruent", "aggregating_zones", package = "spData")
incongr_wgs = st_transform(incongruent, 4326)
open_app()
#> Error: Python shared library not found, Python bindings not loaded.
#> Use reticulate::install_miniconda() if you'd like to install a Miniconda Python environment.
Created on 2020-01-14 by the reprex package (v0.3.0)
Guess Python install is at fault here...
Update @jannes-m I forgot about RQGIS 3 which fixes it:
devtools::install_github("jannes-m/RQGIS3") library(sf) library(raster) library(RQGIS3) set_env(dev = FALSE) data("incongruent", "aggregating_zones", package = "spData") incongr_wgs = st_transform(incongruent, 4326) open_app()
find_algorithms("union", name_only = TRUE)
Update, I get another unexpected minor error here continuing with RQGIS3:
alg = "qgis:union"
open_help(alg)
# Error in open_help(alg) : The specified algorithm qgis:union does not exist.
But the next line works:
get_usage(alg)
Union (native:union)
This algorithm checks overlaps between features within the Input layer and creates separate features for overlapping and non-overlapping parts. The area of overlap will create as many identical overlapping features as there are features that participate in that overlap.
An Overlay layer can also be used
in which case features from each layer are split at their overlap with features from the other one
creating a layer containing all the portions from both Input and Overlay layers. The attribute table of the Union layer is filled with attribute values from the respective original layer for non-overlapping features
and attribute values from both layers for overlapping features.
----------------
Input parameters
----------------
INPUT: Input layer
Parameter type: QgsProcessingParameterFeatureSource
Accepted data types:
- str: layer ID
- str: layer name
- str: layer source
- QgsProcessingFeatureSourceDefinition
- QgsProperty
- QgsVectorLayer
OVERLAY: Overlay layer
Parameter type: QgsProcessingParameterFeatureSource
Accepted data types:
- str: layer ID
- str: layer name
- str: layer source
- QgsProcessingFeatureSourceDefinition
- QgsProperty
- QgsVectorLayer
OUTPUT: Union
Parameter type: QgsProcessingParameterFeatureSink
Accepted data types:
- str: destination vector file
e.g. d:/test.shp
- str: memory: to store result in temporary memory layer
- str: using vector provider ID prefix and destination URI
e.g. postgres:... to store result in PostGIS table
- QgsProcessingOutputLayerDefinition
- QgsProperty
----------------
Outputs
----------------
OUTPUT: <QgsProcessingOutputVectorLayer>
Union
Update, this fixes it:
alg = "native:union"
But then this fails:
union = run_qgis("native:union", INPUT = incongr_wgs, INPUT2 = aggzone_wgs,
OUTPUT = file.path(tempdir(), "union.shp"),
load_output = TRUE)
Error in pass_args(alg, ..., params = params, NA_flag = NA_flag, qgis_env = qgis_env) : 'INPUT2' is/are (an) invalid function argument(s). 'native:union' allows following function arguments: 'INPUT', 'OVERLAY', 'OUTPUT'
Fixed by:
union = run_qgis("native:union", INPUT = incongr_wgs, OVERLAY = aggzone_wgs,
OUTPUT = file.path(tempdir(), "union.shp"),
load_output = TRUE)
I feel a PR brewing...
Another update, this yields nothing:
find_algorithms("sliver", name_only = TRUE)
Complete list of GIS algorithms on that docker image (no SAGA):
In fact, I have already started re-writing the R-GIS bridges chapter in a new branch a while ago but then stopped doing so since not all QGIS geoalgorithms I have used in the first edition are also available in QGIS3. At the moment I would rather opt to just take out the RQGIS(3) section in a new edition since there are so many problems with RQGIS3 (RStudio issue on Unix-based systems and starting from QGIS 3.14.13 GRASS and SAGA geoalgorithms are not working any longer, and I have no idea why this is (oth the native QGIS geoalgorithms are still working). And with the twins I am unfortunately missing the time to look for hours and hours in the source code what they might changed in the QGIS API from one version to the next... at least atm this is really difficult.
How about we leave it as-is and aim to update the section in time for the second edition of the book? It seems likely that upstream developments in QGIS and RStudio may automatically resolve the issues by then in any case...
Heads-up @jannes-m I just got a message asking when we plan to update this chapter. I vote by end of 2021 is a reasonable aim for hitting the https://github.com/Robinlovelace/geocompr/milestone/7 milestone and possibly even the 2nd edition. On a related note, what are your thoughts on raster @Nowosad, still too early to replace the raster code?
I hope I'll find some time during my parental leave in June/July this year to work on the R-GIS bridges chapter. I suppose we have to also rewrite the RSAGA section since RSAGA is no longer maintained (though it might still be working).
I have an idea to test different R approaches to raster data during the summer and then decide which one is the best in terms of (a) ease of use, (b) teachability, (c) performance, (d) capabilities. Currently I can see a lot of pros and cons of raster, terra, and stars...
Despite the truly admirable performance of the protagonists, I have been struggling for quite some time in my daily work, both in research and teaching, with (a) the problems of the competing R package concepts, especially of course the terra/raster
and stars/sf
problems, (b) the hardly comprehensible, say, diversity of the migration of the libraries and reference systems etc. on which r-spatial is based. (e.g. but not only gdal/proj
) and (c) the growing difficulties to reliably access the APIs of the big open source software packages (GRASS
as well as QGIS
and ''SAGA'' anyway).
Scientific work is only possible in reliable production systems under reproducible conditions. Both research and teaching (especially remotely with 20 unknown laptops) are a disaster under these conditions, eaten up by the hassle and frustration of fixing problems on a daily basis. That is a real pity.
In my opinion, the problem exceeds the R-GIS bridges chapter and also what some of us can do as developers in addition to the usual activities. Regarding the textbook, I think you should generally consider whether the bridging chapter can or should be maintained in the future. The work on it will be considerable for the foreseeable future. I'm curious what you have in mind, but maybe it's more important to integrate openEO
or similar cloud-oriented concepts instead? The future for most analytics will probably not be on the local workstation.... Anyway if I can support you - gladly.
Thanks for the insights @gisma, really interesting. Re raster packages my thinking is if in doubt sticking to the more mature stuff (raster
in this case) is often safest. It's a question of timing and issues being ironed out I guess. Look forward to insights from @Nowosad on the most efficient (or perhaps least painful) option seems to be.
Ease of use must be a key consideration when remembering that the entire world, including people who have only ever used GUIs, is the target audience. Ease of use partly explains the enduring and IMO justified popularity of the tidyverse
.
Re b) and c) could you try the following?
docker run -d -p 8786:8787 -e DISABLE_AUTH=TRUE -v $(pwd):/home/rstudio/geocompr geocompr/geocompr:qgis
Navigating to http://localhost:8786/ after that incantation I see this (need to add qgisprocess
to the docker image but :tada: nonetheless):
There's also the qgis-ext
image that deserves more prominent mention and which could be a foundation for an update bridges chapter: https://github.com/geocompr/docker/pull/8
Update on this @gisma this works well, giving 970 algorithms, an embarrassment of riches!
docker run -d -p 8786:8787 -e DISABLE_AUTH=TRUE -v $(pwd):/home/rstudio/geocompr geocompr/geocompr:qgis-ext
@Robinlovelace i totally agree with you! Thanks for the docker hint. It is downloading.... The docker approach is obviously one of the most promising ways out of the installation and dependencies jungle.
Closing this for now because it seems to work and, more importantly, it is partly superseded by #519.
Can you try to reproduce this @jannes-m:
And then in that container's RStudio Server instance:
Created on 2020-01-14 by the reprex package (v0.3.0)
Session info
``` r devtools::session_info() #> Warning in system("timedatectl", intern = TRUE): running command 'timedatectl' #> had status 1 #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 3.6.2 (2019-12-12) #> os Debian GNU/Linux 10 (buster) #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz Etc/UTC #> date 2020-01-14 #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date lib source #> assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.2) #> backports 1.1.5 2019-10-02 [1] CRAN (R 3.6.2) #> callr 3.4.0 2019-12-09 [1] CRAN (R 3.6.2) #> class 7.3-15 2019-01-01 [2] CRAN (R 3.6.2) #> classInt 0.4-2 2019-10-17 [1] CRAN (R 3.6.2) #> cli 2.0.1 2020-01-08 [1] CRAN (R 3.6.2) #> codetools 0.2-16 2018-12-24 [2] CRAN (R 3.6.2) #> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.2) #> DBI 1.1.0 2019-12-15 [1] CRAN (R 3.6.2) #> desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.2) #> devtools 2.2.1 2019-09-24 [1] CRAN (R 3.6.2) #> digest 0.6.23 2019-11-23 [1] CRAN (R 3.6.2) #> e1071 1.7-3 2019-11-26 [1] CRAN (R 3.6.2) #> ellipsis 0.3.0 2019-09-20 [1] CRAN (R 3.6.2) #> evaluate 0.14 2019-05-28 [1] CRAN (R 3.6.2) #> fansi 0.4.1 2020-01-08 [1] CRAN (R 3.6.2) #> fs 1.3.1 2019-05-06 [1] CRAN (R 3.6.2) #> glue 1.3.1 2019-03-12 [1] CRAN (R 3.6.2) #> highr 0.8 2019-03-20 [1] CRAN (R 3.6.2) #> hms 0.5.3 2020-01-08 [1] CRAN (R 3.6.2) #> htmltools 0.4.0 2019-10-04 [1] CRAN (R 3.6.2) #> jsonlite 1.6 2018-12-07 [1] CRAN (R 3.6.2) #> KernSmooth 2.23-16 2019-10-15 [2] CRAN (R 3.6.2) #> knitr 1.26 2019-11-12 [1] CRAN (R 3.6.2) #> lattice 0.20-38 2018-11-04 [2] CRAN (R 3.6.2) #> magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.2) #> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.6.2) #> pillar 1.4.3 2019-12-20 [1] CRAN (R 3.6.2) #> pkgbuild 1.0.6 2019-10-09 [1] CRAN (R 3.6.2) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 3.6.2) #> pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.6.2) #> prettyunits 1.1.0 2020-01-09 [1] CRAN (R 3.6.2) #> processx 3.4.1 2019-07-18 [1] CRAN (R 3.6.2) #> ps 1.3.0 2018-12-21 [1] CRAN (R 3.6.2) #> R6 2.4.1 2019-11-12 [1] CRAN (R 3.6.2) #> rappdirs 0.3.1 2016-03-28 [1] CRAN (R 3.6.2) #> raster * 3.0-10 2020-01-13 [1] Github (rspatial/raster@0c39ab5) #> Rcpp 1.0.3 2019-11-08 [1] CRAN (R 3.6.2) #> readr 1.3.1 2018-12-21 [1] CRAN (R 3.6.2) #> remotes 2.1.0 2019-06-24 [1] CRAN (R 3.6.2) #> reticulate * 1.14 2019-12-17 [1] CRAN (R 3.6.2) #> rgdal 1.4-8 2019-11-27 [1] CRAN (R 3.6.2) #> rlang 0.4.2 2019-11-23 [1] CRAN (R 3.6.2) #> rmarkdown 2.0 2019-12-12 [1] CRAN (R 3.6.2) #> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.2) #> RQGIS * 1.0.4 2018-08-13 [1] CRAN (R 3.6.2) #> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.6.2) #> sf * 0.8-1 2020-01-14 [1] Github (r-spatial/sf@1e095d4) #> sp * 1.3-2 2019-11-07 [1] CRAN (R 3.6.2) #> stringi 1.4.5 2020-01-11 [1] CRAN (R 3.6.2) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 3.6.2) #> testthat 2.3.1 2019-12-01 [1] CRAN (R 3.6.2) #> tibble 2.1.3 2019-06-06 [1] CRAN (R 3.6.2) #> units 0.6-5 2019-10-08 [1] CRAN (R 3.6.2) #> usethis 1.5.1 2019-07-04 [1] CRAN (R 3.6.2) #> vctrs 0.2.1 2019-12-17 [1] CRAN (R 3.6.2) #> withr 2.1.2 2018-03-15 [1] CRAN (R 3.6.2) #> xfun 0.12 2020-01-13 [1] CRAN (R 3.6.2) #> yaml 2.2.0 2018-07-25 [1] CRAN (R 3.6.2) #> zeallot 0.1.0 2018-01-28 [1] CRAN (R 3.6.2) #> #> [1] /usr/local/lib/R/site-library #> [2] /usr/local/lib/R/library ```