Open Nic-Ma opened 2 years ago
Hi, in a MONAI deploy WG meeting last week @ericspod mentioned that it might be useful to comment here on experiences with other app packaging frameworks. I just wanted to share my experiences with the MLFlow python_function "flavor", Unfortunately, I'm not able to share all the source code at this time, but I'll try to describe it best I can with a short excerpt. The model itself is an NLP application for classifying endoscopy reports based on transformers pre-trained BERT model.
The basis of the package is this wrapper class which just requires __init__
and a predict
methods. At the end of my training script, an instance of this class is created using an input model (in this case a LightningModule
) and the tokenizer. This wrapped model object is then logged to mlflow where it can be deployed through mlflow's model serving framework.
"""
mlflow python_model wrapper class for Barretts model
"""
import mlflow
import pandas as pd
from torch import topk
from project.BarrettsDataset import BarrettsDataset
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
class BarrettsWrapper(mlflow.pyfunc.PythonModel):
def __init__(self, model, tokenizer):
self.model = model
self.tokenizer = tokenizer
self.dataset = BarrettsDataset(tokenizer=tokenizer)
def predict(self, context, model_input):
logger.info('Running prediction service ')
encoding = self.dataset.encoder(model_input.diag_final.tolist())
_, test_prediction = self.model(encoding["input_ids"], encoding["attention_mask"])
res = topk(test_prediction, 1).indices.tolist()
confidences = pd.DataFrame(test_prediction.tolist(), columns=self.dataset.label_columns)
prediction = pd.DataFrame({'Predicition': [self.dataset.label_columns[x[0]] for x in res]})
results = pd.concat([prediction, confidences], axis=1)
return results
# *** training code goes here, "model" is trained LightningModule, test_df is example input dataframe ***
wrappedModel = BarrettsWrapper(model, tokenizer)
signature = mlflow.models.signature.infer_signature(test_df, wrappedModel.predict(None, test_df))
mlflow.pyfunc.log_model('barretts_nlp', python_model=wrappedModel, signature=signature, code_path=['.'])
Some things I think might be useful to mention:
I hope some of this rambling is useful, these are just a few things which come to mind, very happy to discuss if you have any questions!
Hi @laurencejackson ,
Thanks so much for detailed sharing and feedback! About the things you mentioned:
verification schema
in our proposal, I put into the task 5 and task 6 of https://github.com/Project-MONAI/MONAI/issues/3482 and already implemented a network verification example in: https://github.com/Project-MONAI/tutorials/pull/487.Thanks.
Is your feature request related to a problem? Please describe. Sub task of ticket https://github.com/Project-MONAI/MONAI/issues/3482 Create an example of model package.