How to create a layer with additional packages

[dorkalev]

hi, everything works great for me on lambda I am trying to follow your guidelines and add four new libraries to my lambda: biostrings, httr, stringr and deployr

i am creating a new layer with this script ... (i run it after build_recommend...)

!/bin/bash

set -euo pipefail rm -rf R/ mkdir -p R/library cd R/library wget https://cran.r-project.org/src/contrib/httr_1.4.0.tar.gz wget https://bioconductor.org/packages/release/bioc/src/contrib/Biostrings_2.50.2.tar.gz wget https://cran.r-project.org/src/contrib/stringr_1.4.0.tar.gz wget https://cran.r-project.org/src/contrib/dplyr_0.8.0.1.tar.gz ls .tar.gz | xargs -i tar -xvzf {} rm .tar.gz cd .. cd ..

chmod -R 755 R/ rm -f dorke.zip zip -r -q dorke.zip R/

later i tell lambda to use the main layer (runtime) and this new layer it's not working

would appreciate assistance

[dorkalev]

ok, i think i get it - i need to actually install the packages and take the compiled folders, not just the source as i did <

[philipp94831]

Hi, yes you need to use the compiled package files. I will update the documentation accordingly. Good to hear that it seems to be working nonetheless :-)

[LGuilhas]

Hi Guys,

In my case I hit another problem with the size of the libraries and dependencies exceeding the max 250 MB for the runtime layers size.

I had to workaround that by putting the libraries on s3 and the create an initialization mechanism that fetches the zip from s3 and extracts it to the tmp folder.

As lambdas reuse containers this only afects the first run after a cold start but doesn’t impact performance afterwars.

I’ll gladly share my initialization script if it helps someone (note: I had never written a line of R before this).

[nathmenini]

Hi all.

Everything works great for me on lambda too. But I am still trying to figure out how could I create a new layer containing some libraries: magrittr, mlr, ...

Could someone help me do this? I will be eternally grateful ;)

[philipp94831]

Hi @nathmenini, with the recent Docker support for compiling R and layers it has become much easier to build your layer. I updated the README:

You can use Docker for building your layer. You need to run ./docker_build.sh first. Then you can install your packages inside the container and copy the files to your machine. See awspack/ for an example. The build.sh script is used to run the docker container and copy sources to your machine. The entrypoint.sh script is used for installing packages inside the container.

I hope this helps

[ngfrey]

@philipp94831 Is there a full working example of adding new packages and building out a working lambda layer? I'm still new to docker, so I went with the EC2 approach. Is the simplest way of adding more packages just to add them to the install.packages line in aws-lambda-r-runtime/r/compile.sh?

[philipp94831]

Hi @ngfrey, I would definitely recommend going the docker way. If you don't want to, you can use https://github.com/bakdata/aws-lambda-r-runtime/blob/master/awspack/compile.sh as a starting point for packaging your own libraries. You should use the Lambda AMI with an appropriate R installation.