search

north-road / qgis-processing-r

QGIS Processing R Provider Plugin
https://north-road.github.io/qgis-processing-r/
GNU General Public License v3.0
63 stars 14 forks source link

Reading or writing vectors other than ESRI Shape not possible #48

Open Bernd-Vogelgesang opened 4 years ago

Bernd-Vogelgesang commented 4 years ago

Hi, took me a while till I figured out, that the R-provider does not handle input other that ESRI Shape files. Loading gpkg or sqlite failed. The same applies to the output. Took me again a while to figure out that I have to manually pick shp and save to file to get results.

Linux Mint 19.2, QGIS 3.10

nyalldawson commented 4 years ago

That shouldn't be the case -- there's multiple unit tests covering over formats here too.

Can you share your script?

Bernd-Vogelgesang commented 4 years ago

Hi Nyall, I stripped down my script to crucial parts:

input=vector

result=output vector

result<-input

With a gpkg as input:

Linking to sp version: 1.3-1
Fehler in ogrInfo(dsn = dsn, layer = layer, encoding = encoding, use_iconv = use_iconv, :
Cannot open layer
Ruft auf: readOGR -> suppressMessages -> withCallingHandlers -> ogrInfo
Ausführung angehalten

with the same layer, saved as shp:

Linking to sp version: 1.3-1
OGR data source with driver: ESRI Shapefile
Source: "/home/bernd/Schreibtisch/temp", layer: "test3"
with 9 features
It has 3 fields
Integer64 fields read as strings: length bereich
Fehler in writeOGR(result, "/tmp/processing_e03aaf7cc83343538484dcd42bf8c17a/567640844abf458b8a4fb39988416e0f/result.gpkg", :
Creating Name field failed
Ausführung angehalten

with the shape input and out manually set to shp:

[1] "/home/bernd/R/x86_64-pc-linux-gnu-library/3.6/rgdal"
[1] "/home/bernd/R/x86_64-pc-linux-gnu-library/3.6/raster"
Lade nötiges Paket: sp
rgdal: version: 1.4-7, (SVN revision 845)
Geospatial Data Abstraction Library extensions to R successfully loaded
Loaded GDAL runtime: GDAL 2.4.2, released 2019/06/28
Path to GDAL shared files: /usr/share/gdal
GDAL binary built with GEOS: TRUE
Loaded PROJ.4 runtime: Rel. 5.2.0, September 15th, 2018, [PJ_VERSION: 520]
Path to PROJ.4 shared files: (autodetected)
Linking to sp version: 1.3-1
OGR data source with driver: ESRI Shapefile
Source: "/home/bernd/Schreibtisch/temp", layer: "test3"
with 9 features
It has 3 fields
Integer64 fields read as strings: length bereich
Execution completed in 1.93 seconds
Results:
{'result': '/home/bernd/Schreibtisch/temp/result.shp'}

So, the only successful approach was to take an ESRI Shape as input and also save to file as ESRI Shape. The Input layer is a small line layer with 9 lines. Hope this helps.

nyalldawson commented 4 years ago

This works correctly for me. Here's the full log I get:

options("repos"="http://cran.at.r-project.org/")
.libPaths("/home/nyall/.local/share/QGIS/QGIS3/profiles/default/processing/rlibs")
tryCatch(find.package("rgdal"), error=function(e) install.packages("rgdal", dependencies=TRUE))
tryCatch(find.package("raster"), error=function(e) install.packages("raster", dependencies=TRUE))
library("raster")
library("rgdal")
input=readOGR("/tmp/processing_22300635cb7342e6bba0da7c89933c79/91b6ddb2ad0940df8fef5d32239d8499/result.gpkg")
result<-input

writeOGR(result,"/tmp/processing_22300635cb7342e6bba0da7c89933c79/34858adcab844a609688f68da06bfc2a/result.gpkg","result", driver="GPKG")
R execution console output
[1] "/home/nyall/.local/share/QGIS/QGIS3/profiles/default/processing/rlibs/rgdal"
[1] "/home/nyall/.local/share/QGIS/QGIS3/profiles/default/processing/rlibs/raster"
Loading required package: sp
rgdal: version: 1.4-7, (SVN revision 845)
Geospatial Data Abstraction Library extensions to R successfully loaded
Loaded GDAL runtime: GDAL 2.3.2, released 2018/09/21
Path to GDAL shared files: /usr/share/gdal
GDAL binary built with GEOS: TRUE
Loaded PROJ.4 runtime: Rel. 5.2.0, September 15th, 2018, [PJ_VERSION: 520]
Path to PROJ.4 shared files: (autodetected)
Linking to sp version: 1.3-1
OGR data source with driver: GPKG
Source: "/tmp/processing_22300635cb7342e6bba0da7c89933c79/91b6ddb2ad0940df8fef5d32239d8499/result.gpkg", layer: "result"
with 17 features
It has 6 fields
Execution completed in 0.92 seconds
Results:
{'result': '/tmp/processing_22300635cb7342e6bba0da7c89933c79/e8384d8ce961482f9d34a62da07fe4c1/result.gpkg'}

You'll need to check the first half of your log window, where the actual R script is listed (under "R execution commands"). Try running that script in a standalone R session and see if you can identify whether the readOGR command works correctly there.

Bernd-Vogelgesang commented 4 years ago

Hi Nyall, sorry to steal your time. I did as you told, copied that upper part of a failing run within QGIS to RStudio.

For one dataset, all ran through with a result, but next dataset was failling again. For reproducibility, I now used the easter-egg world_map.gpkg.

QGIS version: 3.10.0-A Coruña
QGIS code revision: 6c816b4204
Qt version: 5.9.5
GDAL version: 2.4.2
GEOS version: 3.7.1-CAPI-1.11.1 27a5e771
PROJ version: Rel. 5.2.0, September 15th, 2018
Processing algorithm…
Algorithm 'test' starting…
Input parameters:
{ 'input' : '/usr/share/qgis/resources/data/world_map.gpkg|layername=countries', 'result' : 'TEMPORARY_OUTPUT' }

R execution commands
options("repos"="http://cran.at.r-project.org/")
.libPaths("/home/bernd/R/x86_64-pc-linux-gnu-library/3.6")
tryCatch(find.package("rgdal"), error=function(e) install.packages("rgdal", dependencies=TRUE))
tryCatch(find.package("raster"), error=function(e) install.packages("raster", dependencies=TRUE))
library("raster")
library("rgdal")
input=readOGR("/tmp/processing_02b3576a20d14575b9e1feeeb773eef1/a559c362aca94bdbb0f475c4eb9c4e73",layer="input")
result<-input
writeOGR(result,"/tmp/processing_02b3576a20d14575b9e1feeeb773eef1/95238aa426094b2995f8959801a6366c/result.gpkg","result", driver="GPKG")
R execution console output
[1] "/home/bernd/R/x86_64-pc-linux-gnu-library/3.6/rgdal"
[1] "/home/bernd/R/x86_64-pc-linux-gnu-library/3.6/raster"
Lade nötiges Paket: sp
rgdal: version: 1.4-7, (SVN revision 845)
Geospatial Data Abstraction Library extensions to R successfully loaded
Loaded GDAL runtime: GDAL 2.4.2, released 2019/06/28
Path to GDAL shared files: /usr/share/gdal
GDAL binary built with GEOS: TRUE
Loaded PROJ.4 runtime: Rel. 5.2.0, September 15th, 2018, [PJ_VERSION: 520]
Path to PROJ.4 shared files: (autodetected)
Linking to sp version: 1.3-1
OGR data source with driver: ESRI Shapefile
Source: "/tmp/processing_02b3576a20d14575b9e1feeeb773eef1/69611382c79841ed98fc1843b6a9ad76", layer: "input"
with 240 features
It has 7 fields
Fehler in writeOGR(result, "/tmp/processing_02b3576a20d14575b9e1feeeb773eef1/07b2837832c94c0ba3231da3a550a9dd/result.gpkg", :
Creating Name field failed
Ausführung angehalten
Execution completed in 2.39 seconds
Results:
{'result': '/tmp/processing_02b3576a20d14575b9e1feeeb773eef1/07b2837832c94c0ba3231da3a550a9dd/result.gpkg'}

Loading resulting layers
The following layers were not correctly generated.<ul><li>/tmp/processing_02b3576a20d14575b9e1feeeb773eef1/95238aa426094b2995f8959801a6366c/result.gpkg</li></ul>You can check the 'Log Messages Panel' in QGIS main window to find more information about the execution of the algorithm.

This also fails within Rstudio with "Creating Name field failed"

Recreating this in Rstudio manually works just fine.

ogrInfo("/usr/share/qgis/resources/data/world_map.gpkg")
ogrListLayers("/usr/share/qgis/resources/data/world_map.gpkg")
input <- readOGR("/usr/share/qgis/resources/data/world_map.gpkg",layer="countries")
result <- input
writeOGR(result,"/home/bernd/Schreibtisch/temp/result.gpkg","result", driver="GPKG")

So I assume that there must be something going on within the R-provider which leads to these problems.

for completeness, my sessioninfo()


sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Linux Mint 19.2

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=de_DE.UTF-8       LC_NUMERIC=C               LC_TIME=de_DE.UTF-8        LC_COLLATE=de_DE.UTF-8     LC_MONETARY=de_DE.UTF-8   
 [6] LC_MESSAGES=de_DE.UTF-8    LC_PAPER=de_DE.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rgdal_1.4-7  raster_3.0-7 sp_1.3-1    

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.2       codetools_0.2-16 lattice_0.20-38  MASS_7.3-51.4    grid_3.6.1       arm_1.10-1       nlme_3.1-141     stats4_3.6.1    
 [9] coda_0.19-3      mi_1.0           minqa_1.2.4      nloptr_1.2.1     Matrix_1.2-17    boot_1.3-23      splines_3.6.1    lme4_1.1-21     
[17] tools_3.6.1      yaml_2.2.0       abind_1.4-5      compiler_3.6.1