search

r-spatial / sf

Simple Features for R
https://r-spatial.github.io/sf/
Other
1.34k stars 298 forks source link

Error reading .kmz file from vignette #133

Closed pssguy closed 7 years ago

pssguy commented 7 years ago 
library(sf)
u_kmz <- "http://coagisweb.cabq.gov/datadownload/BikePaths.kmz"download.file(u_kmz, "BikePaths.kmz") 

biketrails_kmz <- st_read("BikePaths.kmz")
Error in CPL_read_ogr(dsn, layer, as.character(options), quiet, iGeomField -  : 
Open failed.

 sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

locale:
[1] LC_COLLATE=English_Canada.1252  LC_CTYPE=English_Canada.1252    LC_MONETARY=English_Canada.1252 LC_NUMERIC=C                   
[5] LC_TIME=English_Canada.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] sf_0.2-7

loaded via a namespace (and not attached):
[1] DBI_0.5-1   tools_3.3.1 Rcpp_0.12.8 grid_3.3.1

I can confirm the BikePaths.kmz is in folder

TIA

etiennebr commented 7 years ago

It works for me on version 0.2-8 from development. It plots nicely ! If you can, install it:

library(devtools)
install_github("edzer/sfr")

Otherwise, if you have gdal installed, you could try system('ogr2ogr -f "ESRI Shapefile" BikePaths.shp BikePaths.kmz') and read st_read("BikePaths.shp").

pssguy commented 7 years ago

@etiennebr Thanks . I'm on 0.2-7 from CRAN - but then so is the vignette!

Tried installing the dev version 

Error : .onAttach failed in attachNamespace() for 'sf', details:
  call: .Call("sf_CPL_proj_version", PACKAGE = "sf", b)
  error: "sf_CPL_proj_version" not available for .Call() for package "sf"

Also get the same, 'Error in CPL_read_ogr' with a different .shp file I wanted to load

edzer commented 7 years ago

I get no problems reading this file with 0.2-7. here it says that "KML reading is only available if GDAL/OGR is built with the Expat XML Parser, otherwise only KML writing will be supported." I see

> st_drivers()[row.names(st_drivers()) == "KML",]
    name                     long_name write  copy is_raster is_vector
KML  KML Keyhole Markup Language (KML)  TRUE FALSE     FALSE      TRUE

what do you see there?

pssguy commented 7 years ago

I get the same result. as you. The vignettes say the assumption is all drivers can read. I'm pretty naive in this area and not sure where the CPL_read_ogr function comes from

BTW I moved and renamed my shp (and other associated files) and that now does not appear to be a problem

edzer commented 7 years ago

Are you sure the .kmz file actually downloaded? The code segment in your issue does not execute, as it is. As a check, could you tell me how large the downloaded file is?

pssguy commented 7 years ago

Hope this helps

bikeissue

dieza commented 7 years ago

Same happens to me on mac. biketrails_kmz <- st_read("BikePaths.kmz") Cannot open data source /Users/agus/BikePaths.kmz Error in CPL_read_ogr(dsn, layer, as.character(options), quiet, iGeomField - : Open failed.

But

ogr2ogr -f "ESRI Shapefile" BikePaths.shp BikePaths.kmz

and

st_read("BikePaths.shp")

works

Reading layer BikePaths' from data source/Users/agus/BikePaths.shp' using driver `ESRI Shapefile' Simple feature collection with 3324 features and 12 fields geometry type: GEOMETRY dimension: XYZ bbox: xmin: -106.7982 ymin: 34.78517 xmax: -106.2212 ymax: 35.37142 epsg (SRID): 4326 proj4string: +proj=longlat +datum=WGS84 +no_defs

sessionInfo

R version 3.3.2 (2016-10-31) Platform: x86_64-apple-darwin13.4.0 (64-bit) Running under: macOS Sierra 10.12.1

locale: [1] ca_ES.UTF-8/ca_ES.UTF-8/ca_ES.UTF-8/C/ca_ES.UTF-8/ca_ES.UTF-8

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] sp_1.2-3 RPostgreSQL_0.4-1 DBI_0.5-1 sf_0.2-7

loaded via a namespace (and not attached): [1] rgdal_1.1-10 tools_3.3.2 Rcpp_0.12.8 grid_3.3.2 lattice_0.20-34

edzer commented 7 years ago

What does

library(rgdal)
x = readOGR("BikePaths.kmz", "BikePaths")

give you?

pssguy commented 7 years ago

@edzer I get


> x = readOGR("BikePaths.kmz", "BikePaths")
Error in ogrInfo(dsn = dsn, layer = layer, encoding = encoding, use_iconv = use_iconv,  : 
  Cannot open data source
``
edzer commented 7 years ago

Looking again, on my machine reading reports

Reading layer `BikePaths' from data source `/tmp/BikePaths.kmz' using driver `LIBKML'
...

so not driver KML but driver LIBKML; what do you see when you ask

> st_drivers()[row.names(st_drivers()) == "LIBKML",]

?

pssguy commented 7 years ago 

> st_drivers()[row.names(st_drivers()) == "LIBKML",]
[1] name      long_name write     copy      is_raster is_vector
<0 rows> (or 0-length row.names)
edzer commented 7 years ago

That's what I suspected. What you may want to try is:

  1. rename BikePaths.kmz to BikePaths.kml.zip
  2. unzip this file
  3. read the unzipped file with st_read("doc.kml")

I think that reading .kmz (i.e.. zipped .kml's) files requires libkml (which your gdal lib doesn' t have), and that the kml driver (not the libkml driver) only reads .kml.

pssguy commented 7 years ago

The unzipping does not work after renaming

"BikePaths.kml.zip is invalid"

edzer commented 7 years ago

From here on, the problems are not related to the sf package, so I suggest to close the issue.

pssguy commented 7 years ago

OK so are you saying kml files cannot be loaded without gdal which is not available for Windows?

edzer commented 7 years ago

Windows (and Mac) binary CRAN packages don't link to external user software (your link to gdal), but come with statically linked gdal/geos/proj, which explains the large size of the .dll in the package lib. This prepackaged gdal does not have libkml (as documented), and hence does not read .kmz files. You can either

  1. rename the .kmz, unzip to .kml, and read with the KML (not LIBKML) driver, as explained above or
  2. install gdal yourself and install sf from source, linking to your gdal installation at install time.
pssguy commented 7 years ago

Thanks for your time As mentioned above, 1 did not work for me re unzipping I'll report on 2 if and when

etiennebr commented 7 years ago

I think the vignette should be fixed, or there should be a note, because this situation will likely happen a hundred times given Windows and CRAN popularity :)

dieza commented 7 years ago

Sorry for catching up a little later. BikePaths.kmz can be read on my mac but unzipping allows to read the result kml.

x<-readOGR("BikePaths.kmz","Bikepaths")

Error in ogrInfo(dsn = dsn, layer = layer, encoding = encoding, use_iconv = use_iconv, : Cannot open data source R version 3.3.2 (2016-10-31) Platform: x86_64-apple-darwin13.4.0 (64-bit) Running under: macOS Sierra 10.12.2

locale: [1] ca_ES.UTF-8/ca_ES.UTF-8/ca_ES.UTF-8/C/ca_ES.UTF-8/ca_ES.UTF-8

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] rgdal_1.2-5 sp_1.2-4 sf_0.2-8 readODS_1.6.4 wordcloud_2.5 RColorBrewer_1.1-2 SnowballC_0.5.1
[8] tm_0.6-2 NLP_0.1-9

loaded via a namespace (and not attached): [1] Rcpp_0.12.8 lattice_0.20-34 assertthat_0.1 slam_0.1-40 grid_3.3.2 cellranger_1.1.0 DBI_0.5-1
[8] units_0.4-1 xml2_1.1.0 tools_3.3.2 udunits2_0.13 readr_1.0.0 parallel_3.3.2 tibble_1.2

El 6Jan, 2017, a las 2:23 AM, Etienne B. Racine notifications@github.com escribi�:

I think the vignette should be fixed, or there should be a note, because this situation will likely happen a hundred times given Windows and CRAN popularity :)

� You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

edzer commented 7 years ago

@Robinlovelace could you pls take a look and suggest an update to the vignette?

Robinlovelace commented 7 years ago

Sure - my bad, wanted to add a real world data set so it's not purely theoretical.

To summarize the situation: will use a .kml file for benchmarking instead - plan to find a kml file from the internet and use that instead.

A wider question: I think there is a need to set-up a directory of example spatial data files. Discussed this with @Nowosad. So could kill 2 birds with one stone by creating such a data repo, perhaps eventually to be submitted to CRAN for people wanting explore spatial data types and datasets and do benchmarking. That way we're not relying on other peoples' servers (which could go down) and could have a spatial equivalent of the datasets package http://stat.ethz.ch/R-manual/R-devel/library/datasets/html/datasets-package.html and providing a range of data formats, like these examples from the rio package: https://github.com/leeper/rio/tree/master/inst/examples

Sound good? Happy to do a quick fix to close this and then look to compile a collection of small yet diverse spatial datasets for benchmarking and other purposes. If something like this exists already please let me know so I don't reinvent any wheels.

edzer commented 7 years ago

Good idea. This would be a separate R package, right?

rsbivand commented 7 years ago

I've been approached by Debian people before about something like this, because of uncertainties about licencing. Having a separate package with example data sets would follow this up. The attempt that I began is at https://r-forge.r-project.org/scm/viewvc.php/pkg/spDatasets/?root=rspatial with datasets from spdep and maptools (maybe more) in RData file format and sp classes.

Robinlovelace commented 7 years ago

Fantastic, many thanks for sharing that existing package Roger. I could add to it rather than create another one - sound good? Or would it be more advantageous to create another package focussed on, but not limited to, file formats, e.g. called spDataForms.

rsbivand commented 7 years ago

You are very welcome to build on this if it is at all useful. The idea was to collect data sets and pre-package them as sp objects to avoid having to read them in each time, but I never got as far as releasing it and shifting spdep and maptools to suit.

These are quotations from an email exchange in 2014:

The purpose of all the packages I maintain is teaching at graduate levels, and all the examples (so the data sets) are included in the packages to permit users to reproduce text book results. This software is not an application, it is intended to permit learning, first by doing what text books say, then learning from own data. This is my ontology. Consequently, the licensing/copyright of the code and its documentation is orthogonal to that of the data sets.

Note that spdep has far more data than maptools, but this has not been questioned. If the data license issues in spdep are not a problem, why were they a problem in maptools? Is GPL a relevant license for data sets (rhetorical question)? The examples on the help pages all need access to data sets - the data sets chosen by me are those that are most relevant for relating the methods to text books in which they are discussed.

One (bad/ugly) resolution is to create a separate data set package. However, this is a general problem because most CRAN packages face the same problem.

By the way, I maintain maptools, and cannot recall being contacted in 2009. Nicholas contributed code in 2000/2001, but now has nothing to do with maptools.

The issue was that Debian required all files to be free to change, whereas I suggested that changing the content of data files makes no great sense. I ended up in my draft package with a CC licence.

Robinlovelace commented 7 years ago

Sounds great - will aim to build on that one. Only issue, I seem incapable of cloning the svn repo, even with this: https://gist.github.com/philchalmers/7281255#file-rforge2git-r-L9 - any chance you could dump it on GitHub or something similar - may just be my account's not set-up yet on R-Forge.

pavelsjo commented 4 years ago

That's what I suspected. What you may want to try is:

1. rename `BikePaths.kmz` to `BikePaths.kml.zip`

2. unzip this file

3. read the unzipped file with `st_read("doc.kml")`

I think that reading .kmz (i.e.. zipped .kml's) files requires libkml (which your gdal lib doesn' t have), and that the kml driver (not the libkml driver) only reads .kml.

Thanks a lot!