r-spatial / sf

Simple Features for R
https://r-spatial.github.io/sf/
Other
1.34k stars 297 forks source link

Unable to install sf on RStudio on Amazon AWS Ubuntu EC2 Cloud #1778

Closed kamaulindhardt closed 3 years ago

kamaulindhardt commented 3 years ago

Hi sf team,

Lately, I have been experimenting with moving some of my computation into Amazon's AWS EC2 platform running RStudio, as it gives flexibility and enormous computer power. Unfortunately, I have several difficulties installing my favorite package sf. Any idea on this issue what might be going on. Looks like my issue is similar to #680. However, I do not know how to solve it based on the thread there. I have no problems installing a number of other packages.

Have you had more experiences with this kind of issue for the sf package on AWS EC2? How are you recommending me to deal with this?

Here is the error warning I get from RStudio

 install.packages("sf")
Installing package into ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
also installing the dependencies ‘e1071’, ‘classInt’

trying URL 'https://cran.rstudio.com/src/contrib/e1071_1.7-8.tar.gz'
Content type 'application/x-gzip' length 581347 bytes (567 KB)
==================================================
downloaded 567 KB

trying URL 'https://cran.rstudio.com/src/contrib/classInt_0.4-3.tar.gz'
Content type 'application/x-gzip' length 403884 bytes (394 KB)
==================================================
downloaded 394 KB

trying URL 'https://cran.rstudio.com/src/contrib/sf_1.0-2.tar.gz'
Content type 'application/x-gzip' length 3645982 bytes (3.5 MB)
==================================================
downloaded 3.5 MB

ERROR: failed to lock directory ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’ for modifying
Try removing ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071’
Warning in install.packages :
  installation of package ‘e1071’ had non-zero exit status
ERROR: dependency ‘e1071’ is not available for package ‘classInt’
* removing ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/classInt’
Warning in install.packages :
  installation of package ‘classInt’ had non-zero exit status
ERROR: dependency ‘classInt’ is not available for package ‘sf’
* removing ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/sf’
Warning in install.packages :
  installation of package ‘sf’ had non-zero exit status

The downloaded source packages are in
    ‘/tmp/Rtmpyuqh99/downloaded_packages’

I have tried to install the packages ‘classInt’ and ‘e1071’ separately, but no luck as it gives me:

 install.packages("e1071")
Installing package into ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/e1071_1.7-8.tar.gz'
Content type 'application/x-gzip' length 581347 bytes (567 KB)
==================================================
downloaded 567 KB

ERROR: failed to lock directory ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’ for modifying
Try removing ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071’
Warning in install.packages :
  installation of package ‘e1071’ had non-zero exit status

The downloaded source packages are in
    ‘/tmp/Rtmpyuqh99/downloaded_packages’
> install.packages("classInt")
Installing package into ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
also installing the dependency ‘e1071’

trying URL 'https://cran.rstudio.com/src/contrib/e1071_1.7-8.tar.gz'
Content type 'application/x-gzip' length 581347 bytes (567 KB)
==================================================
downloaded 567 KB

trying URL 'https://cran.rstudio.com/src/contrib/classInt_0.4-3.tar.gz'
Content type 'application/x-gzip' length 403884 bytes (394 KB)
==================================================
downloaded 394 KB

ERROR: failed to lock directory ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’ for modifying
Try removing ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071’
Warning in install.packages :
  installation of package ‘e1071’ had non-zero exit status
ERROR: dependency ‘e1071’ is not available for package ‘classInt’
* removing ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/classInt’
Warning in install.packages :
  installation of package ‘classInt’ had non-zero exit status

The downloaded source packages are in
    ‘/tmp/Rtmpyuqh99/downloaded_packages’

All the best,

Kamau Lindhardt

edzer commented 3 years ago

Have you tried this?

ERROR: failed to lock directory ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’ for modifying
Try removing ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071’

and then reinstall classInt

kamaulindhardt commented 3 years ago

Thank you @edzer ,

Yes I have tried this. I removed thee1071 package – even though it was never there.. and tried re-installing classInt. Unfortunately no luck..

remove.packages("e1071")
Removing package from ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
Error in remove.packages : there is no package called ‘e1071’
Restarting R session...

> ?e1071
No documentation for ‘e1071’ in specified packages and libraries:
you could try ‘??e1071’
> install.packages(classInt)
Error in install.packages : object 'classInt' not found
> install.packages("calssInt")
Installing package into ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
Warning in install.packages :
  package ‘calssInt’ is not available (for R version 4.0.2)
> install.packages('classInt')
Installing package into ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
also installing the dependency ‘e1071’

trying URL 'https://cran.rstudio.com/src/contrib/e1071_1.7-8.tar.gz'
Content type 'application/x-gzip' length 581347 bytes (567 KB)
==================================================
downloaded 567 KB

trying URL 'https://cran.rstudio.com/src/contrib/classInt_0.4-3.tar.gz'
Content type 'application/x-gzip' length 403884 bytes (394 KB)
==================================================
downloaded 394 KB

ERROR: failed to lock directory ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’ for modifying
Try removing ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071’
Warning in install.packages :
  installation of package ‘e1071’ had non-zero exit status
ERROR: dependency ‘e1071’ is not available for package ‘classInt’
* removing ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/classInt’
Warning in install.packages :
  installation of package ‘classInt’ had non-zero exit status

The downloaded source packages are in
    ‘/tmp/RtmpKoPiK0/downloaded_packages’
edzer commented 3 years ago

What does

ls -la /home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071

give?

kamaulindhardt commented 3 years ago

Appreciate your helping hand.

I have no experience with this command.. Does it go in the Terminal?

It gives me:

rstudio@ip-172-31-3-157:~/ERAgroforestry$ ls -la /home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071
total 20
drwxr-xr-x   3 rstudio rstudio  4096 Sep  2 05:33 .
drwxr-xr-x 283 rstudio rstudio 12288 Sep  2 11:29 ..
drwxr-xr-x   3 rstudio rstudio  4096 Sep  2 05:33 00new
rstudio@ip-172-31-3-157:~/ERAgroforestry$ 

image

image

rsbivand commented 3 years ago

What is the output of .libPaths()? How did you install goftest and randomForest? Can you run any regular command in the console (example(lm) for example)? Are you the rstudio user? If not, you may not have write access - try system("whoami").

kamaulindhardt commented 3 years ago

Thank you @rsbivand ,

Im quite new to running RStudio on Amazon's AWS EC2 virtual machines.. But for a few days now I have been using it to run some relatively simple tidymodels machine learning analysis – and never had problems installing any required packages or loading data. I wanted now to experiment with geo-spatial data (analysis) and ran into the issue with the sf package.

As far as I know, I am the rstudio but not sure about access..

.libPaths()
[1] "/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0"
[2] "/usr/local/lib/R/site-library"                  
[3] "/usr/lib/R/site-library"                        
[4] "/usr/lib/R/library" 

It is basically just running RStudio as I would have it on my personal laptop. No problems running regular commands:

> example(lm) 

lm> require(graphics)

lm> ## Annette Dobson (1990) "An Introduction to Generalized Linear Models".
lm> ## Page 9: Plant Weight Data.
lm> ctl <- c(4.17,5.58,5.18,6.11,4.50,4.61,5.17,4.53,5.33,5.14)

lm> trt <- c(4.81,4.17,4.41,3.59,5.87,3.83,6.03,4.89,4.32,4.69)

lm> group <- gl(2, 10, 20, labels = c("Ctl","Trt"))

lm> weight <- c(ctl, trt)

lm> lm.D9 <- lm(weight ~ group)

lm> lm.D90 <- lm(weight ~ group - 1) # omitting intercept

lm> ## No test: 
lm> anova(lm.D9)
Analysis of Variance Table

Response: weight
          Df Sum Sq Mean Sq F value Pr(>F)
group      1 0.6882 0.68820  1.4191  0.249
Residuals 18 8.7293 0.48496               

lm> summary(lm.D90)

Call:
lm(formula = weight ~ group - 1)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.0710 -0.4938  0.0685  0.2462  1.3690 

Coefficients:
         Estimate Std. Error t value Pr(>|t|)    
groupCtl   5.0320     0.2202   22.85 9.55e-15 ***
groupTrt   4.6610     0.2202   21.16 3.62e-14 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.6964 on 18 degrees of freedom
Multiple R-squared:  0.9818,    Adjusted R-squared:  0.9798 
F-statistic: 485.1 on 2 and 18 DF,  p-value: < 2.2e-16

lm> ## End(No test)
lm> opar <- par(mfrow = c(2,2), oma = c(0, 0, 1.1, 0))

lm> plot(lm.D9, las = 1)      # Residuals, Fitted, ...
Hit <Return> to see next plot: 
lm> par(opar)

lm> ## Don't show: 
lm> ## model frame :
lm> stopifnot(identical(lm(weight ~ group, method = "model.frame"),
lm+                     model.frame(lm.D9)))

lm> ## End(Don't show)
lm> ### less simple examples in "See Also" above
lm> 
lm> 
lm> 

image

The system("whoami")gives me:

system("whoami")
rstudio
There were 12 warnings (use warnings() to see them)
> warnings()
Warning messages:
1: In doTryCatch(return(expr), name, parentenv, handler) :
  display list redraw incomplete
2: In doTryCatch(return(expr), name, parentenv, handler) : invalid graphics state
3: In doTryCatch(return(expr), name, parentenv, handler) : invalid graphics state
4: In doTryCatch(return(expr), name, parentenv, handler) :
  display list redraw incomplete
5: In doTryCatch(return(expr), name, parentenv, handler) : invalid graphics state
6: In doTryCatch(return(expr), name, parentenv, handler) : invalid graphics state
7: In doTryCatch(return(expr), name, parentenv, handler) :
  display list redraw incomplete
8: In doTryCatch(return(expr), name, parentenv, handler) : invalid graphics state
9: In doTryCatch(return(expr), name, parentenv, handler) : invalid graphics state
10: In doTryCatch(return(expr), name, parentenv, handler) :
  display list redraw incomplete
11: In doTryCatch(return(expr), name, parentenv, handler) :
  invalid graphics state
12: In doTryCatch(return(expr), name, parentenv, handler) :
  invalid graphics state
edzer commented 3 years ago

I have no experience with this command.. Does it go in the Terminal?

If you work with an AWS instance, it will pay off getting some experience here, so you understand what error messages mean. The help you ask has nothing to do with sf.

In the terminal, give

rm -fr /home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071

and if that doesn't work

sudo rm -fr /home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071

then install e1071 and the rest.

kamaulindhardt commented 3 years ago

Yes @edzer ,

THANK YOU SO MUCH! Now I get a little more understanding of what was happening.

Edzer Pebesma, enjoy the rest of the Open GeoHub sessions. Hope I did not disturb you too much..

Running sudo rm -fr /home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071

rstudio@ip-172-31-3-157:~/ERAgroforestry$ ls -la /home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071
total 20
drwxr-xr-x   3 rstudio rstudio  4096 Sep  2 05:33 .
drwxr-xr-x 283 rstudio rstudio 12288 Sep  2 11:29 ..
drwxr-xr-x   3 rstudio rstudio  4096 Sep  2 05:33 00new
rstudio@ip-172-31-3-157:~/ERAgroforestry$ sudo rm -fr /home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071
[sudo] password for rstudio: 
Sorry, try again.
[sudo] password for rstudio: 
rstudio@ip-172-31-3-157:~/ERAgroforestry$ ls -la /home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071
ls: cannot access '/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071': No such file or directory
rstudio@ip-172-31-3-157:~/ERAgroforestry$ 

Then I restarted the R session and successfully installed e1071 and classInt

Restarting R session...

> install.packages("e1071")
Installing package into ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/e1071_1.7-8.tar.gz'
Content type 'application/x-gzip' length 581347 bytes (567 KB)
==================================================
downloaded 567 KB

* installing *source* package ‘e1071’ ...
** package ‘e1071’ successfully unpacked and MD5 sums checked
** using staged installation
checking for C++ compiler default output file name... a.out
checking whether the C++ compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ -std=gnu++11 accepts -g... yes
** libs
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG      -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-ttHamR/r-base-4.0.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c Rsvm.c -o Rsvm.o
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG      -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-ttHamR/r-base-4.0.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c cmeans.c -o cmeans.o
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG      -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-ttHamR/r-base-4.0.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c cshell.c -o cshell.o
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG      -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-ttHamR/r-base-4.0.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c floyd.c -o floyd.o
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG      -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-ttHamR/r-base-4.0.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c init.c -o init.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG      -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-ttHamR/r-base-4.0.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c svm.cpp -o svm.o
g++ -std=gnu++11 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o e1071.so Rsvm.o cmeans.o cshell.o floyd.o init.o svm.o -L/usr/lib/R/lib -lR
installing to /home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-e1071/00new/e1071/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (e1071)

The downloaded source packages are in
    ‘/tmp/RtmpSc5bY4/downloaded_packages’
> install.packages("classInt")
Installing package into ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/classInt_0.4-3.tar.gz'
Content type 'application/x-gzip' length 403884 bytes (394 KB)
==================================================
downloaded 394 KB

* installing *source* package ‘classInt’ ...
** package ‘classInt’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
gfortran -fno-optimize-sibling-calls  -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-ttHamR/r-base-4.0.2=. -fstack-protector-strong  -c fish1.f -o fish1.o
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG      -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-ttHamR/r-base-4.0.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c init.c -o init.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o classInt.so fish1.o init.o -lgfortran -lm -lquadmath -L/usr/lib/R/lib -lR
installing to /home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-classInt/00new/classInt/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (classInt)

The downloaded source packages are in
    ‘/tmp/RtmpSc5bY4/downloaded_packages’

And finally the sf package

> install.packages("sf")
Installing package into ‘/home/rstudio/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/sf_1.0-2.tar.gz'
Content type 'application/x-gzip' length 3645982 bytes (3.5 MB)
==================================================
downloaded 3.5 MB

* installing *source* package ‘sf’ ...
** package ‘sf’ successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: gcc -std=gnu99
configure: CXX: g++ -std=gnu++11
checking for gdal-config... /usr/bin/gdal-config
checking gdal-config usability... yes
configure: GDAL: 2.2.3
checking GDAL version >= 2.0.1... yes

[.....]

installing to /home/rstudio/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-sf/00new/sf/libs
** R
** demo
** inst
** byte-compile and prepare package for lazy loading
in method for ‘dbWriteTable’ with signature ‘"PostgreSQLConnection","character","sf"’: no definition for class “PostgreSQLConnection”
in method for ‘dbDataType’ with signature ‘"PostgreSQLConnection","sf"’: no definition for class “PostgreSQLConnection”
in method for ‘coerce’ with signature ‘"Spatial","sf"’: no definition for class “Spatial”
in method for ‘coerce’ with signature ‘"Spatial","sfc"’: no definition for class “Spatial”
in method for ‘coerce’ with signature ‘"sf","Spatial"’: no definition for class “Spatial”
in method for ‘coerce’ with signature ‘"sfc","Spatial"’: no definition for class “Spatial”
in method for ‘coerce’ with signature ‘"XY","Spatial"’: no definition for class “Spatial”
in method for ‘coerce’ with signature ‘"crs","CRS"’: no definition for class “CRS”
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (sf)

The downloaded source packages are in
    ‘/tmp/RtmpSc5bY4/downloaded_packages’