search

r-spatial / sf

Simple Features for R
https://r-spatial.github.io/sf/
Other
1.33k stars 294 forks source link

Problem with proj when installing sf #1557

Closed Memochipan closed 3 years ago

Memochipan commented 3 years ago

I apologize if this is not a problem directly with sf but I'm not able to install it due a problem with proj that I don't know how to solve and appreciate your suggestions.

`install.packages('sf', repos='https://cran.rstudio.com/') --2020-12-11 17:20:17-- https://cran.rstudio.com/src/contrib/sf_0.9-6.tar.gz Connecting to 168.176.239.40:8080... connected. Proxy request sent, awaiting response... 200 OK Length: 4160758 (4.0M) [application/x-gzip] Saving to: “/tmp/RtmpOev6Vu/downloaded_packages/sf_0.9-6.tar.gz”

100%[==========================================================>] 4,160,758 --.-K/s in 0.06s

2020-12-11 17:20:17 (66.5 MB/s) - “/tmp/RtmpOev6Vu/downloaded_packages/sf_0.9-6.tar.gz” saved [4160758/4160758]

The downloaded source packages are in ‘/tmp/RtmpOev6Vu/downloaded_packages’ Updating HTML index of packages in '.Library' Making 'packages.html' ... done Warning message: In install.packages("sf", repos = "https://cran.rstudio.com/") : installation of package ‘sf’ had non-zero exit status`

edzer commented 3 years ago

Sounds like you have multiple versions of PROJ / proj.4 installed on your system.

Memochipan commented 3 years ago

Excuse my ignorance. How did you notice that? Which one should I keep in this case in order to make it works? I think I installed twice from different sources trying to make it function.

edzer commented 3 years ago

The proj.cpp compilation errors indicate that ./configure detected one version of PROJ, but header files of another PROJ version were found, which don't match. Entirely removing either of them should work. Note that GDAL also needs PROJ.

Memochipan commented 3 years ago

Thank you. You were right. I had multiple versions of PROJ. I removed one of them via yum and kept the conda one.

But, it seems that the error is something else:

`install.packages('sf', repos='https://cran.rstudio.com/') Installing package into ‘/home/relexteriores/R/x86_64-conda_cos6-linux-gnu-library/3.6’ (as ‘lib’ is unspecified) --2020-12-14 11:44:54-- https://cran.rstudio.com/src/contrib/sf_0.9-6.tar.gz Connecting to 168.176.239.40:8080... connected. Proxy request sent, awaiting response... 200 OK Length: 4160758 (4.0M) [application/x-gzip] Saving to: “/tmp/Rtmp5nvfFJ/downloaded_packages/sf_0.9-6.tar.gz”

100%[==========================================================>] 4,160,758 --.-K/s in 0.03s

2020-12-14 11:44:54 (150 MB/s) - “/tmp/Rtmp5nvfFJ/downloaded_packages/sf_0.9-6.tar.gz” saved [4160758/4160758]

The downloaded source packages are in ‘/tmp/Rtmp5nvfFJ/downloaded_packages’ Warning message: In install.packages("sf", repos = "https://cran.rstudio.com/") : installation of package ‘sf’ had non-zero exit status`

What else I'm missing here?

edzer commented 3 years ago

It looks like this: https://trac.osgeo.org/gdal/wiki/BuildingOnUnixGDAL25dev : some header file (GDAL?) renamed all PROJ symbols. As this is the first time I see this happening in an sf build, my guess is that it has something to do with conda/GDAL/PROJ setup. Please ask for help by those who maintain that, and report back here when solved.

Note that conda+R is not an officially supported R distro or channel; you may think it is great that it (to some extent) works, but it means that those who support R (like me) not necessarily support conda+R, in particular for the kind of problems you report above which are neither caused by R nor by sf, nor by all the usual GDAL / PROJ installs we've seen.

Memochipan commented 3 years ago

Ok. I got the path of trying to install in other clean machine and again the bottle neck happened when trying to install the sf package due the GDAL version under 2.0.

To avoid problems I followed this guide not related to conda: https://thelinuxcluster.com/2019/12/17/compile-gdal-2-4-3-on-centos-7/

But when installing sf with this command:

install.packages('sf', type = "source", configure.args='--with-gdal-config=/usr/local/gdal-2.4.3/bin/gdal-config --with-proj-lib=/usr/local/proj-5.2.0/lib')

I get this error:

checking GDAL: checking whether PROJ is available for linking:... yes checking GDAL: checking whether PROJ is available fur running:... ./gdal_proj: error while loading shared libraries: libgdal.so.20: cannot open shared object file: No such file or directory no configure: error: OGRCoordinateTransformation() does not return a coord.trans: PROJ not available? ERROR: configuration failed for package ‘sf’

I had nightmares installing this package and it is the only step missing for the installation of what we need that is the mapview.

Thank you in advice.

edzer commented 3 years ago

Why would you install old gdal and proj versions on a clean machine?

On December 24, 2020 7:21:50 PM GMT+01:00, Memochipan notifications@github.com wrote:

Ok. I got the path of trying to install in other clean machine and again the bottle neck happened when trying to install the sf package due the GDAL version under 2.0.>

To avoid problems I followed this guide not related to conda:> https://thelinuxcluster.com/2019/12/17/compile-gdal-2-4-3-on-centos-7/>

But when installing sf with this command:>

install.packages('sf', type = "source", configure.args='--with-gdal-config=/usr/local/gdal-2.4.3/bin/gdal-config --with-proj-lib=/usr/local/proj-5.2.0/lib')>

I get this error:>

checking GDAL: checking whether PROJ is available for linking:... yes> checking GDAL: checking whether PROJ is available fur running:... ./gdal_proj: error while loading shared libraries: libgdal.so.20: cannot open shared object file: No such file or directory> no> configure: error: OGRCoordinateTransformation() does not return a coord.trans: PROJ not available?> ERROR: configuration failed for package ‘sf’>

I had nightmares installing this package and it is the only step missing for the installation of what we need that is the mapview.>

Thank you in advice.>

-- > You are receiving this because you commented.> Reply to this email directly or view it on GitHub:> https://github.com/r-spatial/sf/issues/1557#issuecomment-750940409

Memochipan commented 3 years ago

Well, that's a good point. I just found that guide for CentOS 7 that is the machine that have available. I'll try with newer versions and come back to tell if that works.

Memochipan commented 3 years ago

Same result using the latest versions of GDAL and PROJ:

install.packages('sf', type = "source", configure.args='--with-gdal-config=/usr/local/gdal-3.2.0/bin/gdal-config --with-proj-lib=/usr/local/proj-7.2.0/lib')

The error:

checking GDAL: checking whether PROJ is available for linking:... yes checking GDAL: checking whether PROJ is available fur running:... ./gdal_proj: error while loading shared libraries: libgdal.so.28: cannot open shared object file: No such file or directory no configure: error: OGRCoordinateTransformation() does not return a coord.trans: PROJ not available? ERROR: configuration failed for package ‘sf’

edzer commented 3 years ago

Sounds like you installed gdal without (or before?) Proj.

On December 25, 2020 4:06:30 PM GMT+01:00, Memochipan notifications@github.com wrote:

Same result using the latest versions of GDAL and PROJ:>

install.packages('sf', type = "source", configure.args='--with-gdal-config=/usr/local/gdal-3.2.0/bin/gdal-config --with-proj-lib=/usr/local/proj-7.2.0/lib')>

The error:>

checking GDAL: checking whether PROJ is available for linking:... yes> checking GDAL: checking whether PROJ is available fur running:... ./gdal_proj: error while loading shared libraries: libgdal.so.28: cannot open shared object file: No such file or directory> no> configure: error: OGRCoordinateTransformation() does not return a coord.trans: PROJ not available?> ERROR: configuration failed for package ‘sf’>

-- > You are receiving this because you commented.> Reply to this email directly or view it on GitHub:> https://github.com/r-spatial/sf/issues/1557#issuecomment-751260749

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

Memochipan commented 3 years ago

Not really. I installed PROJ using:

./configure --prefix=/usr/local/proj-7.2.0
make -j 16
make install

And then GDAL linked to PROJ:

./configure --prefix=/usr/local/gdal-3.2.0 --with-proj=/usr/local/proj-7.2.0 LDFLAGS="-L/usr/local/proj-7.2.0/lib" 2
CPPFLAGS="-I/usr/local/proj-7.2.0/include" --with-threads --with-libtiff=internal --with-geotiff=internal --with-jpeg=internal --with-gif=internal --with-png=internal --with-libz=internal
make -j 16
make install

I overpassed that error adding the path of libgdal.so to ld.so.conf and running ldconfig.

Now it appears a new error:

configure: GDAL: 3.2.0
checking proj.h usability... no
checking proj.h presence... no
checking for proj.h... no
checking proj_api.h usability... no
checking proj_api.h presence... no
checking for proj_api.h... no
configure: error: proj_api.h not found in standard or given locations.
ERROR: configuration failed for package ‘sf’

Even specifying the include path:

install.packages("sf", configure.args=c("--with-gdal-config=/usr/local/gdal-3.2.0/bin/gdal-config", "--with-proj-lib=/usr/local/proj-7.2.0/lib", "--with-proj-include=/usr/local/proj-7.2.0/include"))

Is there an error in the syntaxis or maybe a bug in the code that didn't recognize the provided path?

Memochipan commented 3 years ago

I have tried from the command line:

sudo R CMD INSTALL sf_0.9-6.tar.gz --configure-args='--with-gdal-config=/usr/local/gdal-3.2.0/bin/gdal-config --with-proj-include=/usr/local/proj-7.2.0/include --with-proj-lib=/usr/local/proj-7.2.0/lib'

And from GitHub:

install_github("r-spatial/sf", configure.args="--with-gdal-config=/usr/local/gdal-3.2.0/bin/gdal-config --with-proj-lib=/usr/local/proj-7.2.0/lib --with-proj-include=/usr/local/proj-7.2.0/include")

Always same error:

checking GDAL: /usr/local/gdal-3.2.0/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.2.0
checking proj.h usability... no
checking proj.h presence... no
checking for proj.h... no
checking proj_api.h usability... no
checking proj_api.h presence... no
checking for proj_api.h... no
configure: error: proj_api.h not found in standard or given locations.
ERROR: configuration failed for package ‘sf’

Path of the resource:

locate proj_api.h
/usr/local/proj-7.2.0/include/proj_api.h
edzer commented 3 years ago

The problems you have are with non-standard setups of GDAL and PROJ. In the README a number of options are suggested to install on a variety of linux platforms. You seem to want to do something different. Since there's no one here to help you with GDAL and PROJ, I'm afraid you need to sort this out yourself.

Memochipan commented 3 years ago

Thank you for nothing. It seems you put little effort to do this project better. It's a shame and a waste of time try to get help here.

edzer commented 3 years ago

"This project"? This is an R package that can be installed without problem from source once GDAL, PROJ and GEOS have been properly installed. Suggestions how to do this have been giving. You're expecting help for setting up GDAL and PROJ in a non-standard way. That is not the problem "this project" tries to solve. Please also note that this project has a code of conduct.

Memochipan commented 3 years ago

I apologize if my words were rude, but I reviewed each of your answers and it seems litlle effort trying to understand the problem or pointing to get a effective solution. All of them are just saying "this is not a problem with sf".

I'm here just because need to install a Shiny Server in CentOS 7 with the mapview package and sf is one of the dependencies.

All the other libraries installed relatively easy but I'm now stuck with sf for three weeks.

I needed to install gdal and proj from source because the required versions are not available for CentOS 7 using the suggested command.

Now I'm trying to link sf to those libraries but it doesn't recognize them in any of the tried alternatives.

I think the problem is with sf recognizing the libraries and that's a valid question here.

tim-salabim commented 3 years ago

@Memochipan a couple of thoughts from my side:

  1. I think you might have more luck asking such a question on R-Sig-Geo mailing list. Many more people will see your question there and the chances that someone has experience with CentOS 7 is higher. Plus, github issues should really only be used for bug reports and feature requests for the software in question. General questions/help/guidance should be sought after elsewhere (mailing lists or stackoverflow for example).

  2. May I ask what mapview functionality you need that leaflet does not provide (disclaimer, I am the mapview developer)? leaflet does not import any of the spatial R libs that link to system libraries like GDAL or PROJ, so installing should be easier. That said, you may still find that you need any of those spatial R libs to get your data into your app (unless you work with points represented as long/lat columns in a data.frame). But you might want to think about pursuing a different pathway to get your app up and running...

  3. Personally, I think it is rude to show your frustration in the way you did in a repository that provides free open source software. After all, no one here has ever claimed that things work everywhere and that you will get full support for all the problems you may encounter. IMHO @edzer has gone to lengths to provide at least basic information about how to get sf installed on a variety of linux distros (but we cannot expect to cover them all). A quick google suggests that CentOS 7 is more than a decade old, so maybe it is another approach to update that server.

rsbivand commented 3 years ago

mapview uses GDAL and PROJ through sf, fubction st_transform() to transform/project input geometries to those used by leaflet etc. for generating interactive web maps. Having installed PROJ from source, always run make check and ensure that only one copy of all of PROJ exists in your system. Once you know that PROJ works, install GEOS, and run Make check. Only when PROJ and GEOS are installed uniquely, remove all traces of any earlier GDAL installations, and begin installing a fresh GDAL. Try running ogrinfo on a vector file at the prompt to see whether GDAL may be missing any shared objects. After installing each library, run ldconfig to check that the shared objects are on the search path. Install all of these in the default location (/usr/local). Install the sf tarball downloaded from CRAN at the command prompt (then you get all of the messages) Consider running R CMD check on the sf tarball before R CMD INSTALL.

As @tim-salabim says, Centos 7 is past end of life, and PROJ/GDAL have recently been through major upgrades. You should aim for at least PROJ 6.3.1 with GDAL 3.0.4. And indeed, R-sig-geo is a proper location, an issue is not.

olcaysah commented 3 years ago

Hi All,

@edzer and @tim-salabim thank you for creating such a great libraries. I used both libraries a lot and they are really written very well. @Memochipan I had exactly same issue around 6 months ago and this problem does not related with the "sf" library. This is all related with CentOS 7. If you can, I suggest to change your with the recent OS, because you will have other problems when installing other R libraries. Regarding to this issue, I was able to solve this problem installing previous versions of GDAL and PROJ. In my case following versions worked for me:

IN the following machine: 

Operating System: CentOS Linux 7 (Core)
CPE OS Name: cpe:/o:centos:centos:7
Kernel: Linux 3.10.0-1127.8.2.el7.x86_64
Architecture: x86-64

R:

R version 3.6.0 (2019-04-26) -- "Planting of a Tree"
Copyright (C) 2019 The R Foundation for Statistical Computing
Platform: x86_64-redhat-linux-gnu (64-bit)

PROJ:

wget https://download.osgeo.org/proj/proj-7.0.1.tar.gz
tar xzf proj-7.0.1.tar.gz
cd proj-7.0.1/
sudo ./configure --prefix=/usr/local/proj-7.0.1

GDAL:

curl -L https://download.osgeo.org/gdal/2.4.4/gdal-2.4.4.tar.gz | tar xvz
cd gdal-2.4.4/
./configure
./autogen.sh
sudo make -j$(nproc) && make install

Hope this helps...

Memochipan commented 3 years ago

Thank you for your replies, they were helpful. Specially what @rsbivand wrote about "Install all of these in the default location (/usr/local)" and "run ldconfig".

The solution was really simple. I just copied the files in /usr/local/proj-7.2.0/include to /usr/local/include and the files in /usr/local/proj-7.2.0/lib to /usr/local/lib.

Then I added a .conf file in /etc/ld.so.conf.d with the path to /usr/local/proj-7.2.0/lib and run sudo ldconfig.

It seems that while other packages recognize proj in custom locations, sf doesn't. Particulary, this part does nothing "--with-proj-lib=/usr/local/proj-7.2.0/lib", "--with-proj-include=/usr/local/proj-7.2.0/include"

The command line to install sf was install.packages("sf", configure.args="--with-gdal-config=/usr/local/gdal-3.2.0/bin/gdal-config --with-geos-config=/usr/local/bin/geos-config") without any reference to proj.

No needs to change OS or downgrade libraries.

Again, please excuse my little rudeness, it was useful to catch your attention and get real help that drove to a effective solution.

rsbivand commented 3 years ago

One possibility is that the custom PROJ location confused pkg-config, which would need to be pointed at proj.pc. PROJ needs pkg-config because it doesn't install a dedicated *.config shell script like GEOS and GDAL. I have export PKG_CONFIG_PATH=... in my ~/.bashrc so that:

$ pkg-config proj --libs
-L/usr/local/lib -lproj
arthfen commented 1 year ago

I was having the same issue after compiling PROJ from source, but in my case I didn't have enough privilege to copy files to /usr/local/lib as suggested by @Memochipan.

I ended up following @rsbivand's idea by modifying the PKG_CONFIG_PATH variable, but it didn't work until I created a custom proj.pc file.

Just in case it is helpful for someone in the same situation, the content of ~/.local/lib/pkgconfig/proj.pc was:

prefix=/home/user/.local
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: proj
Description: Custom proj build
Version: 6.3.1
Requires:
Libs.private: 
Libs: -L${libdir} -lproj
Cflags: -I${includedir}/proj

After that, I could install with the custom flags: install.packages('sf', configure.args = c('--with-proj-lib=/home/user/.local/lib --with-proj-include=/home/user/.local/include'))

barracuda156 commented 1 year ago

Oh, this bug never fixed:

checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /opt/local/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... ./configure: line 3660: 15095 Segmentation fault      ./gdal_proj
no
configure: error: OGRCoordinateTransformation() does not return a coord.trans: PROJ not available?
ERROR: configuration failed for package ‘sf’
rsbivand commented 1 year ago

Use only CRAN binaries unless you are able to install from source without help. --static is needed for building CRAN binaries. Note that this issue is closed.

barracuda156 commented 1 year ago

Use only CRAN binaries unless you are able to install from source without help. --static is needed for building CRAN binaries.

CRAN is not an option due to their poor support for anything but a few recent OS versions.

But anyway, I found what configure script disliked – that spatialite was built against a different version of proj than gdal. Fixed, build is fine now.

rgupta2508 commented 1 year ago

Hi @barracuda156 , May i know versions of proj and gdal. i also stuck with same error .

rgupta2508 commented 1 year ago

Use only CRAN binaries unless you are able to install from source without help. --static is needed for building CRAN binaries.

CRAN is not an option due to their poor support for anything but a few recent OS versions.

But anyway, I found what configure script disliked – that spatialite was built against a different version of proj than gdal. Fixed, build is fine now.

Hi @barracuda156 , May i know versions of proj and gdal. i also stuck with same error .

rsbivand commented 1 year ago

@rgupta2508 please do not extend a closed issue that almost certainly is not relevant to your case (legacy macOS installs). If you use macOS, do use CRAN binaries, and for now, wait for the servers to update to 1.0-13 or just install the slightly older binary.

barracuda156 commented 1 year ago

CRAN is not an option due to their poor support for anything but a few recent OS versions. But anyway, I found what configure script disliked – that spatialite was built against a different version of proj than gdal. Fixed, build is fine now.

Hi @barracuda156 , May i know versions of proj and gdal. i also stuck with same error .

@rgupta2508 Sorry, missed your comment.

You can see versions that we use from R-sf details page: https://ports.macports.org/port/R-sf/details https://ports.macports.org/port/gdal (v. 3.7.0) https://ports.macports.org/port/proj (v. 5.2.0) https://ports.macports.org/port/spatialite (v. 5.0.1)

barracuda156 commented 1 year ago

@rgupta2508 please do not extend a closed issue that almost certainly is not relevant to your case (legacy macOS installs). If you use macOS, do use CRAN binaries, and for now, wait for the servers to update to 1.0-13 or just install the slightly older binary.

@rsbivand This is admittedly not my issue, I just bumped into it, but the problem does not seem to be caused by macOS version, rather by version mismatch. Once that is corrected, everything works.

angelosnm commented 8 months ago

Hi All,

@edzer and @tim-salabim thank you for creating such a great libraries. I used both libraries a lot and they are really written very well. @Memochipan I had exactly same issue around 6 months ago and this problem does not related with the "sf" library. This is all related with CentOS 7. If you can, I suggest to change your with the recent OS, because you will have other problems when installing other R libraries. Regarding to this issue, I was able to solve this problem installing previous versions of GDAL and PROJ. In my case following versions worked for me:

IN the following machine:

Operating System: CentOS Linux 7 (Core)
CPE OS Name: cpe:/o:centos:centos:7
Kernel: Linux 3.10.0-1127.8.2.el7.x86_64
Architecture: x86-64

R:

R version 3.6.0 (2019-04-26) -- "Planting of a Tree"
Copyright (C) 2019 The R Foundation for Statistical Computing
Platform: x86_64-redhat-linux-gnu (64-bit)

PROJ:

wget https://download.osgeo.org/proj/proj-7.0.1.tar.gz
tar xzf proj-7.0.1.tar.gz
cd proj-7.0.1/
sudo ./configure --prefix=/usr/local/proj-7.0.1

GDAL:

curl -L https://download.osgeo.org/gdal/2.4.4/gdal-2.4.4.tar.gz | tar xvz
cd gdal-2.4.4/
./configure
./autogen.sh
sudo make -j$(nproc) && make install

Hope this helps...

Do we have any update on those instructions for the newer versions?

barracuda156 commented 8 months ago

@agelosnm You could install from Macports, it should work smoothly on anything from 10.5.8 to 14.2.1.

I have switched all R ports to use Proj 9 too. If something does not work, please open a ticket on Trac and tag me, I will fix it.