search

discindo / r2lambda

Deploy an R script as AWS lambda from the R console
https://discindo.github.io/r2lambda/
GNU General Public License v3.0
33 stars 3 forks source link

Problems with pushing to ECR #11

Open teofiln opened 1 year ago

teofiln commented 1 year ago

As described here https://github.com/richfitz/stevedore/issues/61 and here https://github.com/richfitz/stevedore/issues/63

Maybe call docker externally with a system command while the stevedore issue gets resolved.

DyfanJones commented 1 year ago

Not sure if you're aware, but you can push docker images to aws ecr by using smdocker. It is based off the python cli sm-docker.

A little background information around smdocker. The desired directory containing the dockerfile is zipped up and and pushed to AWS Codebuild to build the docker image. It is then outputs the results to AWS ECR.

The pros with this approach, is that users doesn't require docker to setup on their local computer. Here is an AWS Blog in how it works: https://aws.amazon.com/blogs/machine-learning/using-the-amazon-sagemaker-studio-image-build-cli-to-build-container-images-from-your-studio-notebooks/

teofiln commented 1 year ago

Hi @DyfanJones! Thanks for this! I'll take a look and be in touch.

teofiln commented 1 year ago

Seems like the error Error in log(bytes, 1000) : non-numeric argument to mathematical function that happens sometimes when pushing the image to ECR is down to a function in stevedore that formats the push progress output. I bypased this by removing the calls to the formatting function in my branch of stevedore (here: https://github.com/teofiln/stevedore/blob/38034eb9e4d1a23a06f6e7353dbcdf86edcf09ac/R/docker_client_support.R#L121).

teofiln commented 1 year ago

I'd like to keep the functionality to be able to test the function locally before pushing to ECR. I am not sure how something like this would work if the image is built by smdocker. So, at least for now, we'll keep the system dependency for docker. Maybe later we can add the option to build the image with smdocker, while maintaining the local docker option as well.