Open momegas opened 1 year ago
@stavrostheocharis @gcharis @NickNtamp @sinnec
Here are some thoughts about the implementation of an MLFlow integration. Give your feedback with numbers as below, please. This will help me have other views because I gave a lot of thought to this, I think I'm short-sighted now.
Bump 👋
Bump 💥
Aren't we talking about an mlflow plugin? It makes sense to me
MLFlow plugins are for mlflow to integrate with other tools, right? We need the opposite, I guess. Whitebox should be getting data from mlflow
Some thoughts also from me regarding the above points:
After all these days I think we only need number 1 btw
For anyone taking this issue, lets go with number one option for now:
I propose a method in the SDK that requests the model from MLFlow. Take into account that we may need to point the SDK to MLFlow. Renaming this issue to something more relevant
I think @NickNtamp can share some thoughts on that since he is going to check a bit Mlflow
Based on my investigation till now mlflow is capable of saving/tracking the followings per different experiments:
Based on my knowledge the only thing (at least for now) that can be used is number 2 by replacing the functions which calculate the evaluation metrics here from row 65 and below.
Description
Since MLFlow is an industry standard and a lot of people use it, it makes sense that whitebox integrates with it and uses it as a data store, or something similar providing missing functionality in the monitoring field of MLOps