package BH boost errors env

HannesOberreiter commented 4 years ago

Hi everyone,

I try to build a docker container (FROM rocker/tidyverse:4.0.0) which should use env for version control with caching and the package qpcR.

While installing / compiling the needed packages I run into some trouble, see error message below.

Don't know if this is a problem coming with the ROCKER container.

Thanks! Cheers Hannes

* installing *source* package ‘later’ ...
** package ‘later’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -pthread -DSTRICT_R_HEADERS -I'/home/rstudio/project/renv/library/R-4.0/x86_64-pc-linux-gnu/Rcpp/include' -I'/home/rstudio/project/renv/library/R-4.0/x86_64-pc-linux-gnu/BH/include' -I/usr/local/include   -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -pthread -DSTRICT_R_HEADERS -I'/home/rstudio/project/renv/library/R-4.0/x86_64-pc-linux-gnu/Rcpp/include' -I'/home/rstudio/project/renv/library/R-4.0/x86_64-pc-linux-gnu/BH/include' -I/usr/local/include   -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c callback_registry.cpp -o callback_registry.o
In file included from callback_registry.cpp:1:
/home/rstudio/project/renv/library/R-4.0/x86_64-pc-linux-gnu/BH/include/boost/bind.hpp:22:10: fatal error: boost/bind/bind.hpp: No such file or directory
   22 | #include <boost/bind/bind.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [/usr/local/lib/R/etc/Makeconf:176: callback_registry.o] Error 1
ERROR: compilation failed for package ‘later’
* removing ‘/home/rstudio/project/renv/staging/1/later’
Installing promises [1.1.0] ...
    OK [built from source]
Installing httpuv [1.5.2] ...
    FAILED
Error installing package 'httpuv':

I also added my DOCKERFILE below.

FROM rocker/tidyverse:4.0.0

LABEL maintainer="hoberreiter@gmail.com"

COPY git_config.sh /etc/cont-init.d/gitconfig
COPY . /home/rstudio/project
RUN chown -R rstudio /home/rstudio/project

ENV RENV_VERSION 0.10.0
RUN Rscript -e "install.packages('remotes', repos = c(CRAN = 'https://cloud.r-project.org'))"
RUN Rscript -e "remotes::install_github('rstudio/renv@${RENV_VERSION}')"

# rgl package, used in qpcR package
RUN apt-get update -qq && apt-get -y --no-install-recommends install \
    libglu1-mesa-dev

# httpuv https://github.com/rstudio/shiny/issues/2073
RUN apt-get -y --no-install-recommends install \
    gfortran libreadline6-dev libx11-dev libxt-dev libcairo2-dev libbz2-dev liblzma-dev libcurl4-openssl-dev cmake

RUN mkdir /home/rstudio/.local
RUN mkdir /home/rstudio/.local/share
RUN mkdir /home/rstudio/.local/share/renv
RUN mkdir /home/rstudio/.local/share/renv/binary

# TODO dont chmod a+rwx? Cannot install binaries and project wihout it (env::init())
RUN chmod a+rwx /home/rstudio/.local
RUN chown -R rstudio /home/rstudio/.local

eddelbuettel commented 4 years ago

I do not why the build error of a "randomly chosen" package off CRAN would be a Rocker bug.

Maybe it is renv, maybe it is later, maybe it is something else. I have no idea. I happen to also maintain BH but I see no issue here.

(You could use a little bit of Unix + Docker refinment. Instead of

RUN mkdir /home/rstudio/.local
RUN mkdir /home/rstudio/.local/share
RUN mkdir /home/rstudio/.local/share/renv
RUN mkdir /home/rstudio/.local/share/renv/binary

so a single

RUN mkdir -p /home/rstudio/.local/share/renv/binary

Ditto for

RUN chmod a+rwx /home/rstudio/.local
RUN chown -R rstudio /home/rstudio/.local

which I would write as

RUN chmod a+rwx /home/rstudio/.local \
     && chown -R rstudio /home/rstudio/.local

as each RUN becomes one layer.

That's not your bug though which is a different issue. I also do not understand why one would want renv (which freezes versions) inside Docker (which already does that) but I digress....

Lastly, we offer several venues to take advantage of prebuilt binaries to avoid whatever building issue you may run into...

I suggest we close this as I see no Rocker bug here (yet, so feel free to reformulate if you think Rocker is at fault).

HannesOberreiter commented 4 years ago

Hi,

thanks for the response! Pretty new to the Docker environment and really appreciate your help.

I follow the guide https://rstudio.github.io/renv/articles/docker.html to use renv and docker, because I thinks it is a good idea.

Lastly, we offer several venues to take advantage of prebuilt binaries to avoid whatever building issue you may run into...

Could you give me some hints how i can do this?

Closing this Issue as you said.

Cheers Hannes

eddelbuettel commented 4 years ago

Here is one way:

edd@rob:~$ docker run --rm -ti rocker/r-ubuntu:18.04 bash
root@a70e39f754ff:/# apt update
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]                                                                       
Get:3 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic InRelease [15.4 kB] 
Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [931 kB]                          
Get:5 http://ppa.launchpad.net/marutter/rrutter3.5/ubuntu bionic InRelease [15.4 kB]                                                         
Get:6 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [59.3 kB]                                                                
Get:7 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [8,815 B]                                               
Get:8 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [852 kB]                        
Get:9 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main Sources [1,818 kB]                        
Get:10 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]        
Get:11 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]        
Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Get:13 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1,344 kB]      
Get:15 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]          
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [73.6 kB]    
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,228 kB]          
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [20.1 kB]     
Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,385 kB]      
Get:20 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [8,286 B]         
Get:21 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [8,158 B]     
Get:22 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 Packages [877 kB]    
Get:23 http://ppa.launchpad.net/marutter/rrutter3.5/ubuntu bionic/main Sources [9,707 B]
Get:24 http://ppa.launchpad.net/marutter/rrutter3.5/ubuntu bionic/main amd64 Packages [8,457 B]
Fetched 20.7 MB in 5s (3,765 kB/s)                   
Reading package lists... Done
Building dependency tree       
Reading state information... Done
100 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@a70e39f754ff:/# apt install r-cran-httpuv
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  r-cran-bh r-cran-later r-cran-magrittr r-cran-promises r-cran-r6 r-cran-rcpp r-cran-rlang
The following NEW packages will be installed:
  r-cran-bh r-cran-httpuv r-cran-later r-cran-magrittr r-cran-promises r-cran-r6 r-cran-rcpp r-cran-rlang
0 upgraded, 8 newly installed, 0 to remove and 100 not upgraded.
Need to get 12.4 MB of archives.
After this operation, 133 MB of additional disk space will be used.
Do you want to continue? [Y/n]  
Get:1 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-bh all 1.72.0-3-1cran1ppabionic0 [7,431 kB]
Get:2 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-rcpp amd64 1.0.4.6-1cran1ppabionic0 [1,936 kB]                                                                                   
Get:3 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-r6 all 2.4.1-1cran1ppabionic0 [61.9 kB]                                                                                          
Get:4 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-rlang amd64 0.4.6-1cran1.1804.0 [996 kB]                                                                                         
Get:5 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-later amd64 1.0.0-1cran1ppabionic0 [105 kB]                                                                                      
Get:6 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-magrittr all 1.5-5cran2ubuntu2~ubuntu18.04.1~ppa1 [149 kB]                                                                       
Get:7 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-promises amd64 1.1.0-1cran1ppabionic0 [1,197 kB]                                                                                 
Get:8 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-httpuv amd64 1.5.2-1cran1ppabionic0 [510 kB]                                                                                     
Fetched 12.4 MB in 20s (632 kB/s)                                                                                                                                                                                 
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package r-cran-bh.
(Reading database ... 19372 files and directories currently installed.)
Preparing to unpack .../0-r-cran-bh_1.72.0-3-1cran1ppabionic0_all.deb ...
Unpacking r-cran-bh (1.72.0-3-1cran1ppabionic0) ...
Selecting previously unselected package r-cran-rcpp.
Preparing to unpack .../1-r-cran-rcpp_1.0.4.6-1cran1ppabionic0_amd64.deb ...
Unpacking r-cran-rcpp (1.0.4.6-1cran1ppabionic0) ...
Selecting previously unselected package r-cran-r6.
Preparing to unpack .../2-r-cran-r6_2.4.1-1cran1ppabionic0_all.deb ...
Unpacking r-cran-r6 (2.4.1-1cran1ppabionic0) ...
Selecting previously unselected package r-cran-rlang.
Preparing to unpack .../3-r-cran-rlang_0.4.6-1cran1.1804.0_amd64.deb ...
Unpacking r-cran-rlang (0.4.6-1cran1.1804.0) ...
Selecting previously unselected package r-cran-later.
Preparing to unpack .../4-r-cran-later_1.0.0-1cran1ppabionic0_amd64.deb ...
Unpacking r-cran-later (1.0.0-1cran1ppabionic0) ...
Selecting previously unselected package r-cran-magrittr.
Preparing to unpack .../5-r-cran-magrittr_1.5-5cran2ubuntu2~ubuntu18.04.1~ppa1_all.deb ...
Unpacking r-cran-magrittr (1.5-5cran2ubuntu2~ubuntu18.04.1~ppa1) ...
Selecting previously unselected package r-cran-promises.
Preparing to unpack .../6-r-cran-promises_1.1.0-1cran1ppabionic0_amd64.deb ...
Unpacking r-cran-promises (1.1.0-1cran1ppabionic0) ...
Selecting previously unselected package r-cran-httpuv.
Preparing to unpack .../7-r-cran-httpuv_1.5.2-1cran1ppabionic0_amd64.deb ...
Unpacking r-cran-httpuv (1.5.2-1cran1ppabionic0) ...
Setting up r-cran-rcpp (1.0.4.6-1cran1ppabionic0) ...
Setting up r-cran-r6 (2.4.1-1cran1ppabionic0) ...
Setting up r-cran-bh (1.72.0-3-1cran1ppabionic0) ...
Setting up r-cran-rlang (0.4.6-1cran1.1804.0) ...
Setting up r-cran-magrittr (1.5-5cran2ubuntu2~ubuntu18.04.1~ppa1) ...
Setting up r-cran-later (1.0.0-1cran1ppabionic0) ...
Setting up r-cran-promises (1.1.0-1cran1ppabionic0) ...
Setting up r-cran-httpuv (1.5.2-1cran1ppabionic0) ...
root@a70e39f754ff:/#

HannesOberreiter commented 4 years ago

Thank you very much @eddelbuettel it works now. Build from from source now also works, probably was missing some dependancies.

Need to have a look how I can use the caching from renv in my workflow with the prebuild binaries, but you were a big help!

cboettig commented 4 years ago

@HannesOberreiter can you provide a bit more information about what you are doing, e.g. your renv lockfile? Your Dockerfile above should build just fine, but it does not seem to be trying to install the qpcR package you mention.

Note that you can get binary installation on the image you're using with:

docker run --rm -ti rocker/tidyverse:4.0.0 bash
install2.r qpcR

This works because that image uses a CRAN mirror which has pre-built linux binaries for this distro. You can change the mirror to a snapshot so that you get the same versions each time if you prefer. We do that automatically for you on rocker versioned stack once a newer version of R is released. See https://packagemanager.rstudio.com/client/#/repos/1/overview

Of course this should work with renv too, but I need more context to be able to reproduce the error you show above.

HannesOberreiter commented 4 years ago

@cboettig thank you for your answer. I try to build a workflow were I can use my own renv library cache but still produce reproducible code with smaller size containers.

I had in mind that the libraries should install on runtime renv::restore() and use my cache or if no cache is given it should pull the binaries from CRAN.

At the moment my "work-in-progress" project looks like this: github project with docker file connected to the dockerhub were it builds my github file automatically /r/hannesoberreiter/melt_graz.

I start up the container and link my cache folder, start renv::restore() and it should use the cache or load it from CRAN mirror.

I think there is something happening at the renv project were it should fetch prebuild binaries, but I do not fully understand if this is exactly my problem. https://github.com/rstudio/renv/issues/430

cboettig commented 4 years ago

@HannesOberreiter

RSPM is still quite new and it looks like the dust hasn't settled yet in renv (e.g. the open issue you've linked). Your renv lockfile has RSPM as the repository in some places, though RSPM is not defined at the top of the repositories list in the lockfile, which only defines CRAN. I'm not quite up-to-speed on renv at the moment, but that seems potentially problematic.

HannesOberreiter commented 4 years ago

@cboettig thank you very much for the explanation! It sheds a little bit more light into my dark nights to understand how to best work with docker and rocker. :)

rocker-org / rocker

package BH boost errors env #405