45 deploy to fly - Githubissues

mt7180 / quaigle

RAG-based LLM application: project to explore different llm-frameworks like llamaindex, marvin and langchain, other used frameworks: streamlit, fastapi

1 stars 1 forks source link

45 deploy to fly #45

Closed mt7180 closed 12 months ago

mt7180 commented 1 year ago

wip: currently under construction

status:

frontend is up, reads DEBUG variable from secrets and sets fly-internal url for backend ("http://quaigleapi.internal:8000") and needs only 1 vm (shared-cpu-1x)
backend is up, listens on external and internal 8000 port-> for internal bind ipv6 with Dockerfile command: CMD ["uvicorn", "code.fastapi_app:app", "--host", "::", "--port", "8000", "--reload"] and needs minimum 2 vm (shared-cpu-2x)
backend talks to frontend, but constantly runs out of memory. upload small txt file and answering questions + generating quiz is possible. The same for upload of website url (small wikipedia side) Uploading sqlite database (170MB) runs out of memory
reduced backend requirements significantly without running into marvin ai model meta class conflicts
[ ] check if front or backend runs out of memory

Zaubeerer commented 1 year ago

@mt7180, wow, congratulations!

I would say that you have successfully tested fly, but it may not be the right tool for this job.

So my suggestion is that I give you an intro to pulumi and we set up the backend in AWS with an EC2 instance.

Like such, we can get some beefy machines to run the backend. We can even set it up with prefect such that the EC2 instance gets automatically started and stopped to save money ;-)

Would you like to explore that right now or first tackle the second app and come back to this later

mt7180 commented 1 year ago

@Zaubeerer : This sounds great and learning this could help me with the next app and in general I really want some experience with this ... I'm just not sure how much EC2 instances are free, I set up a Kafka server 2 months ago, so my free trial year started I think ... but nevertheless, I can shut it down and also maybe the app after a while. The prefect option also sounds great 👍

Just for my understanding: is plumi used instead of a GitHub workflow to deploy on aws?

mt7180 commented 1 year ago

@Zaubeerer: Added new job to GitHub Actions workflow to automatically deploy the frontend to fly.io and it worked !!! 🎉 🎉 🎉

Zaubeerer commented 1 year ago

@mt7180, I see you are active here, let me know when I shall review again :)

mt7180 commented 1 year ago

@Zaubeerer : Ready, last change before merging into main would be to change GitHub action with frontend deployment to be executed when pushed on main:

deploy_frontend: if: ${{ github.ref == 'refs/heads/main' }}

[ ] and later after deploying backend to pulumi have to change the API_URL in streamlit_app.py again

mt7180 commented 12 months ago

implemented a GitHub actions check with dorny/paths-filter@v2 wether frontend/ backend files were changed and now the actions for linting and deploying are only triggered if the files in the respective sub-folder were changed 🎉