Closed turn1a closed 3 years ago
Hello @kaemo,
@takikadiri and I are going to meet @yetudada @DmitriiDeriabinQB and @laisbsc this week as part of their feedback program. This very question will arise in the discussion as one of our "hottest " current problem.
Our current solution consists in a mix of 2. and 3. approach:
PipelineML
class of the kedro-mlflow plugin to store the pipeline and its artifacts in mlflow exactly as descibed in your item 2, as part of the discussion with one of your team member @akruszewski , see https://github.com/Galileo-Galilei/kedro-mlflow/issues/16.We are thinking about either:
serve
command more flexiblebut both come with huge maintenance costs on our side, and we have decided to stick to our current "encapsulating API process" until it hits its limits and becomes untractable for a given project.
I guess kedro-server and universal deployer will adress some of these concerns but AFAIK there is no kedro way to serve a pipeline right now. Maybe one member of the kedro team will more concrete elements on this?
Hi @kaemo and @Galileo-Galilei, here would be my approach:
training
pipeline produces a model, you should be able to use whichever tool best suited for your infrastructure to serve your model, including mlflow, tensorflow serving, etc.prediction
pipeline can simply contain the data_engineering
+ feature_engineering
pipelines plus one last node that sends the data to the model's API over HTTP (for example). You can run this with kedro run --pipeline=prediction
Let me know what you think of this approach. cc @yetudada @DmitriiDeriabinQB
I think model scoring naturally falls into that "model deployment" epic, which we did identified, but not yet formalised into anything concrete. Kedro in its current state focuses primarily on batch processing, therefore online inference (especially dependant on some feature engineering steps) is out of its scope.
The approach described in option 3 in the original post makes general sense to me, however, understandably, it requires a considerable effort to integrate all moving parts together since Kedro doesn't have anything like "Kedro Server" publicly supported.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Dear Kedro crew,
We are working on an extended example using Titanic dataset that is going to showcase a reasonably sophisticated, end-to-end Machine Learning project utilising best Software Engineering, Machine Learning and Kedro practices. We are trying to figure out the best way to serve a model for an online inference that is not only making predictions but is also performing some preprocessing steps on the data being sent to the model.
Here’s some background information regarding our architecture.
We have developed three modular pipelines:
data_engineering
feature_engineering
modelling
Each of these pipelines can be created in one of two modes:training
prediction
The idea is to share all required training nodes of the
training
pipeline with theprediction
pipeline. We are usingpredict=True
argument tocreate_pipeline
to inform that we want specifictraining
nodes to be included/excluded inprediction
pipeline (for example we leave outfit_imputer
but includepredict_imputer
). We recently figured out that a more appropriate way to do this would be using tags but didn’t manage to refactor our code yet.At the end of the
modelling
pipeline, we output a model. Our prediction pipeline reuses some training nodes and works very well for batch inference, but we would also like to serve this model for an online inference in a Docker container. For this purpose, we are using MLflow’s pyfunc model wrapping LightGBM, but the model alone is not enough: we need data that goes into the model to go through the steps indata_engineering
andfeature_engineering
.Here we are stuck, and we don’t know what would be the best practice. The possibilities we thought about are:
kedro serve
command. The reasons for that would be:Considering the third option, we imagined that it could be automated (with a plugin), and the whole web server could be generated/filled with project pipelines/hooks/other components. It could be packed into a container using kedro-docker and easily deployed after that. Right now, the third option is a simple idea, and we didn’t dive into the details as we wanted to ask for your advice first.
I hope we just missed something and that there is a kedro way/solution to such a scenario.