search

GeroVanMi / algorithmic-quartet-mlops

A showcase Machine Learning Operations (MLOps) Project.
0 stars 1 forks source link

Decide on Deployment Server #5

Closed TheMadBevah closed 6 months ago

TheMadBevah commented 7 months ago

Possible Options:

Currently leaning towards MLServer or FastAPI.

Important: Decide on why this Framework was chosen.

TheMadBevah commented 7 months ago

However, due to the nature of the MLOps Module, im leaning towards MLServer. It has the possibility of already integrating certain Runtimes and shows the setup in Readme

It is built on FastAPI, so the support with, and has an example for serving a custom model

For now: MLServer, based on FastAPI, with already integrated MLFlow as reasoning. --> Working theory, have to see if possible anyhow.

If the MLServer pays off, the ease of use and creation of a Dockerimage with integrated commands prevents possible issues arising. Also avoids the FastAPI extra step, is instead built upon REST and gRPC interface

Close when done

GeroVanMi commented 6 months ago

I agree on the choice of MLServer, though I wasn't able to find the appropriate documentation to run a pytorch model. As long as you are sure that you can pull that off, I think it is a good idea, since it was mentioned in the lecture.

Regarding the pitch: Are there any concrete benefits that you were able to identify (Less development time / boilerplate code, better docs, greater flexibility of some kind or maybe pre-existing Docker images)? What is the difference between serving a model through just FastAPI vs. MLServer?

Don't spend more than 20 minutes on this, if you can't figure it out and you're still sure that you can provide a PyTorch model with MLServer (easily) then you can just give a heads-up and close this. If you're at all unsure, maybe the simpler approach with FastAPI would be better. There I also already have experience with building a docker container for it.

@TheMadBevah

TheMadBevah commented 6 months ago

irrelevant