search

grycap / scar

Serverless Container-aware ARchitectures (e.g. Docker in AWS Lambda)
https://scar.readthedocs.io/en/latest/
Apache License 2.0
598 stars 46 forks source link

Getting No logs/failures for private docker "scar init -n SREBlackBoxTester -i 8236.dkr.ecr.us-west-2.amazonaws.com/abc-gat:latest" #350

Open tarunkumard opened 4 years ago

tarunkumard commented 4 years ago

Hi Bro, i am trying to execute a private docker image hosted on amazon ECR ,but problem is when i execute it ,i see no logs getting generated ,neither any failure in AWS cloud-watch or scar logs..my suspicion is scar not executing the private docker image on ECR

Below are core steps with which i run or execute my private image ,with scar i am not getting the output in AWS lambda or cloud-watch help or suggest alternate action needed to make this private docker image on AWS lambda

here is the command ~ $ scar init -n SREBlackBoxTester -i 8236.dkr.ecr.us-west-2.amazonaws.com/abc-gat:latest" ~ $ scar log -n SREBlackBoxTester ~ $ scar rm -n SREBlackBoxTester Log group '/aws/lambda/Tester' successfully deleted. Function 'Tester' successfully deleted.

tarunkumard commented 4 years ago

@srisco is it possible to execute private docker on AWS ECR to execute on SCAR ,i have used images from dockerhub ,but i need a private docker image to run serverless on AWS lambda ,possible?

gmolto commented 4 years ago

Hi. Support for pulling images from AWS ECR should be implemented in udocker (not in SCAR). Last time we tested this (a year and a half ago) this support was not available but you can try with the latest update of udocker and let us know if it works.

gmolto commented 4 years ago

For the sake of reference: https://github.com/indigo-dc/udocker/issues/168

adokter commented 4 years ago

Hi @gmolto, any chance pulling from AWS ECR will become an option soon in SCAR? Docker Hub has started to implement pull limits. Until last week, I used to use SCAR to process radar data in real-time, to produce maps of migrating birds at birdcast. Because I'm hitting the new Docker Hub pull limits, using SCAR is no longer an option. I really liked the SCAR implementation over batch processing with AWS Batch (faster, better scaling, cheaper), so would love to migrate back to SCAR once this option is there.

gmolto commented 4 years ago

Hi @adokter. Thank you for letting us know your use case. Perhaps you could use S3 as the storage back-end for the Docker images (see: https://scar.readthedocs.io/en/latest/advanced_usage.html#upload-docker-image-files-using-an-s3-bucket), instead of Docker Hub. Also, if Docker images are small they may fit within the payload (see: https://scar.readthedocs.io/en/latest/advanced_usage.html#upload-slim-docker-image-files-in-the-payload).

AWS ECR support will come as soon as udocker implements it. I saw you already opened an issue there.

Please, let us know how you proceed.

adokter commented 4 years ago

Thanks for the pointers @gmolto, I'll look into it!