search

eddelbuettel / r2u

CRAN as Ubuntu Binaries
https://eddelbuettel.github.io/r2u
247 stars 17 forks source link

sf: WARNING: different compile-time and runtime versions for GEOS found #32

Closed MatthieuStigler closed 1 year ago

MatthieuStigler commented 1 year ago

Hi Dirk

I get some warnings with the r2u version of sf, with the infamous message: WARNING: different compile-time and runtime versions for GEOS found:, which triggered some unusual warnings with st_read (example code taken from help page of st_read).

Solution I found was to install myself sf from CRAN, disabling bspm: bspm::disable(); install.packages(c("sf")). I guess that would be a rather tricky issue to address at the r2u level?

Thanks!

output with r2u sf

library(sf)
#> Linking to GEOS 3.10.2, GDAL 3.0.4, PROJ 6.3.1; sf_use_s2() is TRUE
#> WARNING: different compile-time and runtime versions for GEOS found:
#> Linked against: 3.10.2-CAPI-1.16.0 compiled against: 3.8.0-CAPI-1.13.1
#> It is probably a good idea to reinstall sf, and maybe rgeos and rgdal too
dang::shadowedPackages()
#>    Package                                         LibPath Version n  good
#> 1:    bspm /home/matifou/R/x86_64-pc-linux-gnu-library/4.2 0.4.2.3 2  TRUE
#> 2:    bspm                   /usr/local/lib/R/site-library   0.4.2 2 FALSE

## from ?st_read
nc = st_read(system.file("shape/nc.shp", package="sf"), quiet = TRUE)
wkt = st_as_text(st_geometry(nc[1,]))
st_read(system.file("gpkg/nc.gpkg", package="sf"), wkt_filter = wkt, quiet = TRUE)
#> Warning in CPL_read_ogr(dsn, layer, query, as.character(options), quiet, : GDAL
#> Error 1: SQLite error on SELECT name, ellipsoid_auth_name, ellipsoid_code,
#> prime_meridian_auth_name, prime_meridian_code, area_of_use_auth_name,
#> area_of_use_code, publication_date, deprecated FROM geodetic_datum WHERE
#> auth_name = ? AND code = ?: no such column: area_of_use_auth_name
#> Warning in CPL_read_ogr(dsn, layer, query, as.character(options), quiet, : GDAL
#> Message 1: Unable to parse srs_id '4267' well-known text
#> 'GEOGCS["NAD27",DATUM["North_American_Datum_1927",SPHEROID["Clarke
#> 1866",6378206.4,294.9786982138982,AUTHORITY["EPSG","7008"]],EXTENSION["PROJ4_GRIDS","@conus,@alaska,@ntv2_0.gsb,@ntv1_can.dat"],AUTHORITY["EPSG","6267"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4267"]]'
#> Simple feature collection with 4 features and 14 fields
#> Geometry type: MULTIPOLYGON
#> Dimension:     XY
#> Bounding box:  xmin: -81.91116 ymin: 35.98933 xmax: -80.87086 ymax: 36.58965
#> CRS:           NA
#>    AREA PERIMETER CNTY_ CNTY_ID      NAME  FIPS FIPSNO CRESS_ID BIR74 SID74
#> 1 0.081     1.288  1880    1880   Watauga 37189  37189       95  1323     1
#> 2 0.114     1.442  1825    1825      Ashe 37009  37009        5  1091     1
#> 3 0.199     1.984  1874    1874    Wilkes 37193  37193       97  3146     4
#> 4 0.061     1.231  1827    1827 Alleghany 37005  37005        3   487     0
#>   NWBIR74 BIR79 SID79 NWBIR79                           geom
#> 1      17  1775     1      33 MULTIPOLYGON (((-81.80622 3...
#> 2      10  1364     0      19 MULTIPOLYGON (((-81.47276 3...
#> 3     200  3725     7     222 MULTIPOLYGON (((-81.02057 3...
#> 4      10   542     3      12 MULTIPOLYGON (((-81.23989 3...

Created on 2023-02-19 with reprex v2.0.2

output with CRAN sf

bspm::disable()
install.packages(c("sf"))
#> Installing package into '/home/matifou/R/x86_64-pc-linux-gnu-library/4.2'
#> (as 'lib' is unspecified)
library(sf)
#> Linking to GEOS 3.10.2, GDAL 3.4.3, PROJ 8.2.0; sf_use_s2() is TRUE
dang::shadowedPackages()
#>    Package                                         LibPath Version n  good
#> 1:    bspm /home/matifou/R/x86_64-pc-linux-gnu-library/4.2 0.4.2.3 2  TRUE
#> 2:    bspm                   /usr/local/lib/R/site-library   0.4.2 2 FALSE
#> 3:      sf /home/matifou/R/x86_64-pc-linux-gnu-library/4.2   1.0.9 2  TRUE
#> 4:      sf                         /usr/lib/R/site-library   1.0.9 2  TRUE

## from ?st_read
nc = st_read(system.file("shape/nc.shp", package="sf"), quiet = TRUE)
wkt = st_as_text(st_geometry(nc[1,]))
st_read(system.file("gpkg/nc.gpkg", package="sf"), wkt_filter = wkt, quiet = TRUE)
#> Simple feature collection with 4 features and 14 fields
#> Geometry type: MULTIPOLYGON
#> Dimension:     XY
#> Bounding box:  xmin: -81.91116 ymin: 35.98933 xmax: -80.87086 ymax: 36.58965
#> Geodetic CRS:  NAD27
#>    AREA PERIMETER CNTY_ CNTY_ID      NAME  FIPS FIPSNO CRESS_ID BIR74 SID74
#> 1 0.081     1.288  1880    1880   Watauga 37189  37189       95  1323     1
#> 2 0.114     1.442  1825    1825      Ashe 37009  37009        5  1091     1
#> 3 0.199     1.984  1874    1874    Wilkes 37193  37193       97  3146     4
#> 4 0.061     1.231  1827    1827 Alleghany 37005  37005        3   487     0
#>   NWBIR74 BIR79 SID79 NWBIR79                           geom
#> 1      17  1775     1      33 MULTIPOLYGON (((-81.80622 3...
#> 2      10  1364     0      19 MULTIPOLYGON (((-81.47276 3...
#> 3     200  3725     7     222 MULTIPOLYGON (((-81.02057 3...
#> 4      10   542     3      12 MULTIPOLYGON (((-81.23989 3...

Created on 2023-02-19 with reprex v2.0.2

eddelbuettel commented 1 year ago

@MatthieuStigler Can we start at the top please: Which of the two containers did you use?

First off, builds are incremental: if a package is updated, we get the version of that package as it is then. We do not control library versions.

Second, builds are somewhat "unpredictable" in the sense that most package rely on the underlying RSPM/PPM binary. But just to make everybody's life a bit more, maybe 2% or 3% packages comes as source because RSPM/PPM in their magic does not / cannot build it. At that point I build it from source. But it is bloody hard to align it with RSPM/PPM builds I have no control over.

(Third, BioConductor is always from source.)

So in short if something fails let me know and I can maybe rebuild. If it merely warns that is maybe not worth the effort. Overall maybe we need a 'census' for some packages to ensure they all have consistent library versions. Maybe the geospatial community has such a script.

MatthieuStigler commented 1 year ago

Mmh, not sure what are the containers, I am using it here without a Docker infrastructure, just on my machine (using https://dirk.eddelbuettel.com/cranapt). So I guess yes it is difficult to make the example fully reproducible?

Regarding the gravity of the problem, it is hard to tell, so far I encountered quite a few warnings due to the versions mismatch. Given how central sf is to my workflow and how often the warnings appeared, I felt I wanted to make sure they would not appear again.

I don't have any understanding of the RSPM/PPM, but is it possible that sf requires re-compilation whenever rgeos/rgdal are updated, even if sf itself is not updated?

Thanks!!

eddelbuettel commented 1 year ago

So substitute 'container flavour' for 'Ubuntu flavour': You haven't told me if the 'issue' arose under focal or jammy.

MatthieuStigler commented 1 year ago

got you, sorry about that: this is on focal 20.04

eddelbuettel commented 1 year ago

That was the first. Jammy may be smoother so I would try that.

Overall, and as you describe above, "it's tricky". So at this stage the best we can probably do is catalog it. And/or add some helper scripts to catalogue versions. We can of course query apt too. Do you want to poke around a little?

MatthieuStigler commented 1 year ago

Fair point, I should update at some point.

Mmh, if this is tricky for you, it will be even more for me. I am afraid that besides reporting it, I can't be of much help since I don't know about ppm nor cataloguing :-( I hope at least this post can be useful for later discoverability, if other users encounter the same issue and want to get rid of the warning running bspm::disable(); install.packages(c("sf")).

eddelbuettel commented 1 year ago

Pretty straightforward: RSPM/PPM is where we get (most) pre-compiled (non .deb) packages from, this is documented. We then turn those into r-cran-* with full dependencies depending on what was compiled into them. So to some extend what "they do here" matters which is why I pointed you here.

Cataloguing: That was just shorthand for available.packages() and installed.packages() and friens.

And yes, a local install can always be your local fix. I think I close this now as there is no imminent issue for r2u.

MatthieuStigler commented 1 year ago

sounds good, thanks for your insights!

eddelbuettel commented 1 year ago

Keep me posted if/when you see 'odd' error or warning messages. I think sometimes these may be normal / expected, sometimes they may hint at something. I do build BioConductor in bulk because of their release schedule. Maybe if a few geospatial r2u users stick their heads together we can come up with a 'schema' to do something better (if and when we think it;s needed).

MatthieuStigler commented 1 year ago

Just for further info, a quick example where non-updated sf might actually lead to an error. Note that I have sudo apt update with no pending updates (though update.packages() shows further available packages).

library(sf)
#> Linking to GEOS 3.10.2, GDAL 3.0.4, PROJ 6.3.1; sf_use_s2() is TRUE
#> WARNING: different compile-time and runtime versions for GEOS found:
#> Linked against: 3.10.2-CAPI-1.16.0 compiled against: 3.8.0-CAPI-1.13.1
#> It is probably a good idea to reinstall sf, and maybe rgeos and rgdal too
dang::shadowedPackages()
#>     Package                                          LibPath
#>  1:    bspm /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#>  2:    bspm                    /usr/local/lib/R/site-library
#>  3:    dang /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#>  4:    dang                          /usr/lib/R/site-library
#>  5: effects /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#>  6: effects                          /usr/lib/R/site-library
#>  7:    maps /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#>  8:    maps                          /usr/lib/R/site-library
#>  9:    pryr /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#> 10:    pryr                          /usr/lib/R/site-library
#>                  Version n  good
#>  1: <package_version[1]> 2  TRUE
#>  2: <package_version[1]> 2 FALSE
#>  3: <package_version[1]> 2  TRUE
#>  4: <package_version[1]> 2 FALSE
#>  5: <package_version[1]> 2  TRUE
#>  6: <package_version[1]> 2 FALSE
#>  7: <package_version[1]> 2  TRUE
#>  8: <package_version[1]> 2  TRUE
#>  9: <package_version[1]> 2  TRUE
#> 10: <package_version[1]> 2  TRUE

st_crs("ESRI:102033")
#> Warning in CPL_crs_from_input(x): GDAL Error 1: PROJ:
#> proj_create_from_database: SQLite error on SELECT name,
#> coordinate_system_auth_name, coordinate_system_code, geodetic_crs_auth_name,
#> geodetic_crs_code, conversion_auth_name, conversion_code,
#> area_of_use_auth_name, area_of_use_code, text_definition, deprecated FROM
#> projected_crs WHERE auth_name = ? AND code = ?: no such column:
#> area_of_use_auth_name
#> Error in st_crs.character("ESRI:102033"): invalid crs: ESRI:102033

Created on 2023-02-24 with reprex v2.0.2

bspm::disable()
install.packages(c("sf", "rgeos"))
#> Installing packages into '/home/mstigler/R/x86_64-pc-linux-gnu-library/4.2'
#> (as 'lib' is unspecified)
library(sf)
#> Linking to GEOS 3.10.2, GDAL 3.4.3, PROJ 8.2.0; sf_use_s2() is TRUE
dang::shadowedPackages()
#>     Package                                          LibPath
#>  1:    bspm /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#>  2:    bspm                    /usr/local/lib/R/site-library
#>  3:    dang /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#>  4:    dang                          /usr/lib/R/site-library
#>  5: effects /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#>  6: effects                          /usr/lib/R/site-library
#>  7:    maps /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#>  8:    maps                          /usr/lib/R/site-library
#>  9:    pryr /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#> 10:    pryr                          /usr/lib/R/site-library
#> 11:   rgeos /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#> 12:   rgeos                          /usr/lib/R/site-library
#> 13:      sf /home/mstigler/R/x86_64-pc-linux-gnu-library/4.2
#> 14:      sf                          /usr/lib/R/site-library
#>                  Version n  good
#>  1: <package_version[1]> 2  TRUE
#>  2: <package_version[1]> 2 FALSE
#>  3: <package_version[1]> 2  TRUE
#>  4: <package_version[1]> 2 FALSE
#>  5: <package_version[1]> 2  TRUE
#>  6: <package_version[1]> 2 FALSE
#>  7: <package_version[1]> 2  TRUE
#>  8: <package_version[1]> 2  TRUE
#>  9: <package_version[1]> 2  TRUE
#> 10: <package_version[1]> 2  TRUE
#> 11: <package_version[1]> 2  TRUE
#> 12: <package_version[1]> 2  TRUE
#> 13: <package_version[1]> 2  TRUE
#> 14: <package_version[1]> 2  TRUE

st_crs("ESRI:102033")
#> Coordinate Reference System:
#>   User input: ESRI:102033 
#>   wkt:
#> PROJCRS["South_America_Albers_Equal_Area_Conic",
#>     BASEGEOGCRS["SAD69",
#>         DATUM["South American Datum 1969",
#>             ELLIPSOID["GRS 1967 Modified",6378160,298.25,
#>                 LENGTHUNIT["metre",1]]],
#>         PRIMEM["Greenwich",0,
#>             ANGLEUNIT["Degree",0.0174532925199433]]],
#>     CONVERSION["South_America_Albers_Equal_Area_Conic",
#>         METHOD["Albers Equal Area",
#>             ID["EPSG",9822]],
#>         PARAMETER["Latitude of false origin",-32,
#>             ANGLEUNIT["Degree",0.0174532925199433],
#>             ID["EPSG",8821]],
#>         PARAMETER["Longitude of false origin",-60,
#>             ANGLEUNIT["Degree",0.0174532925199433],
#>             ID["EPSG",8822]],
#>         PARAMETER["Latitude of 1st standard parallel",-5,
#>             ANGLEUNIT["Degree",0.0174532925199433],
#>             ID["EPSG",8823]],
#>         PARAMETER["Latitude of 2nd standard parallel",-42,
#>             ANGLEUNIT["Degree",0.0174532925199433],
#>             ID["EPSG",8824]],
#>         PARAMETER["Easting at false origin",0,
#>             LENGTHUNIT["metre",1],
#>             ID["EPSG",8826]],
#>         PARAMETER["Northing at false origin",0,
#>             LENGTHUNIT["metre",1],
#>             ID["EPSG",8827]]],
#>     CS[Cartesian,2],
#>         AXIS["(E)",east,
#>             ORDER[1],
#>             LENGTHUNIT["metre",1]],
#>         AXIS["(N)",north,
#>             ORDER[2],
#>             LENGTHUNIT["metre",1]],
#>     USAGE[
#>         SCOPE["Not known."],
#>         AREA["South America"],
#>         BBOX[-60,-90,15,-30]],
#>     ID["ESRI",102033]]

Created on 2023-02-24 with reprex v2.0.2

eddelbuettel commented 1 year ago

I just did

$ docker run --rm -ti eddelbuettel/r2u:jammy
# install.r sf rgeos
.... omitting output here ...
root@409741feb9cb:/# R -q 
> library(sf)
Linking to GEOS 3.10.2, GDAL 3.4.1, PROJ 8.2.1; sf_use_s2() is TRUE
> library(rgeos)
Loading required package: sp
rgeos version: 0.6-1, (SVN revision 692)
GEOS runtime version: 3.10.2-CAPI-1.16.0 
Please note that rgeos will be retired during 2023,
plan transition to sf functions using GEOS at your earliest convenience.
GEOS using OverlayNG
Linking to sp version: 1.5-1 
Polygon checking: TRUE 
>

So in short I do not see a problem. Am I missing something?

MatthieuStigler commented 1 year ago

oh sorry, I am not familiar with docker, I will make sure to use that when reporting further issues.

eddelbuettel commented 1 year ago

It's just a simple way to "start from scratch" on a clean slate to check the versions in the repo as opposed to whatever I may have hanging around. And it is fast. Has many things going for it.

eddelbuettel commented 1 year ago

You also have a old version of the dang function because it does not print the version numbers right in your example above. A while back I injected an explicit format() to ensure it does.