Various fixes to enable tempo to pack environment for single models

sakoush commented 3 years ago

This PR adds various fixes that we needed to get an mlflow model(e.g based on pytorch) saved and deployed using Tempo.

The main change is to pack the conda environment if save_env is True even if there is no custom predict function (i.e. BaseModel._user_func is False) defined, which is the case with individual models.

Other changes that are required as well:

Pack the runtime mlserver dependency as well, for example in the case of mlfow runtime we need to pack mlserver-mlflow. Models have platform which we use to specify with platform we use. (ModelFramework.MLFlow -> mlserver-mlflow) and that is defined in MLServerRuntimeEnvDeps.
MLServer supports a numpy codec for request input when the whole input in the (case of an image) is being sent in one go as in the first input. Setting content_type=np in the inference request will trigger that. check: https://github.com/SeldonIO/MLServer/pull/286 .

There is an example to showcase the pytorch model serving as a notebook.

TODO in this PR still:

Make sure that the other notebooks are still functional.

ukclivecox commented 3 years ago

Should we add an associated example with MLFlow?

review-notebook-app[bot] commented 3 years ago

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

SeldonIO / tempo

Various fixes to enable tempo to pack environment for single models #189