dgarnitz / vectorflow

VectorFlow is a high volume vector embedding pipeline that ingests raw data, transforms it into vectors and writes it to a vector DB of your choice.
https://www.getvectorflow.com/
Apache License 2.0
667 stars 47 forks source link

Swap Flask API for FastAPI #78

Open dgarnitz opened 10 months ago

dgarnitz commented 10 months ago

Right now we are running Flask with Gunicorn using PM2. There are a few problems with this:

We should swap Flask for FastAPI and (if possible in one PR) swap PM2 for something more appropriate.

Note: We currently depend on Flask-sql alchemy for our DB ORM. We will need to swap this when migrating away from Flask. This may also require changes to the DB services that are also dependencies for the worker and vdb upload worker

OlivierCavadenti commented 10 months ago

Also FastAPI have an awesome automatic documentation generation : https://fastapi.tiangolo.com/tutorial/metadata/

dgarnitz commented 10 months ago

Thanks for linking that @OlivierCavadenti! Any chance you want to implement this upgrade for us? We would really value your contribution!

OlivierCavadenti commented 9 months ago

Thanks for linking that @OlivierCavadenti! Any chance you want to implement this upgrade for us? We would really value your contribution!

I can check want I can do !

datgerhardt commented 6 months ago

Hi @dgarnitz , I will love take this take on

dgarnitz commented 6 months ago

Go for it!

dgarnitz commented 6 months ago

Let me know if I can help!

datgerhardt commented 6 months ago

Hi @dgarnitz, I was trying to maintain the abstraction of the request to get API Keys and files. I am having issues with handling the files. This is sample representation of the file "<_io.BufferedReader name='api/tests/fixtures/test_text.txt'>"

Also with this approach we will lose better auto docs generation support

dgarnitz commented 5 months ago

@datgerhardt Can you provide me with more context around the bug you ran into? Perhaps some screenshots or a code snippet

david-vectorflow commented 5 months ago

Any update?

datgerhardt commented 5 months ago

Thanks for your patience, I had the Internet outage in Ghana due to a problem with undersea cable for past few weeks