We can go about this in a couple of different ways, while focusing mainly on a serverless type of deployment to minimize costs.
This deployment will consist of two main components:
A static deployment of Llamafile for Llama 3.1 8b Instruct in AWS Lambda which will be exposed with AWS API Gateway. This will not get regularly updated unless there are any major changes from the upstream llamafile binary, hence we can be safely sure that this will be a one time job.
Deploying the Streamlit UI in Streamlit Community Cloud or using our Digital Ocean credits to deploy it as a DO app. We mainly want the changes in the Streamlit UI to get reflected whenever code is pushed out over the main branch, which is well supported by both of these options.
DhanshreeA
commented
3 weeks ago
We can go about this in a couple of different ways, while focusing mainly on a serverless type of deployment to minimize costs.
This deployment will consist of two main components:
A static deployment of Llamafile for Llama 3.1 8b Instruct in AWS Lambda which will be exposed with AWS API Gateway. This will not get regularly updated unless there are any major changes from the upstream llamafile binary, hence we can be safely sure that this will be a one time job.
Deploying the Streamlit UI in Streamlit Community Cloud or using our Digital Ocean credits to deploy it as a DO app. We mainly want the changes in the Streamlit UI to get reflected whenever code is pushed out over the main branch, which is well supported by both of these options.