search

awslabs / llrt

LLRT (Low Latency Runtime) is an experimental, lightweight JavaScript runtime designed to address the growing demand for fast and efficient Serverless applications.
Apache License 2.0
8.11k stars 359 forks source link

Base image for container packaged lambda functions? #217

Closed JonHolman closed 8 months ago

JonHolman commented 8 months ago

Will there be a container base image that we can use instead of public.ecr.aws/lambda/nodejs:20 ?

richarddavison commented 8 months ago

We'll add a Lambda OCI (Container) example as well :)

JonHolman commented 8 months ago

Thanks! I was playing around with copying bootstrap into the al2023 provided image, but got lost adding node_modules.

richarddavison commented 8 months ago

@JonHolman it would be helpful if you could give a bit of context of what are you trying to do? Why are you looking to package LLRT in a container?

JonHolman commented 8 months ago

@richarddavison sure. I think having a container option would assist in making LLRT available for more use cases. In what I am trying to do right now and would like to evaluate if LLRT is an option and would improve performance. I'm trying to run llama.cpp with a small LLM within a lambda function. My simple WIP can be seen at https://github.com/JonHolman/wip-sam-node-llama-cpp

richarddavison commented 8 months ago

@JonHolman when we release a new version, you should be able to deploy a container optimized version of LLRT. This should have significantly lower cold starts compared with adding the once that exists today.

richarddavison commented 8 months ago

Released in https://github.com/awslabs/llrt/releases/tag/v0.1.11-beta