search

dotnet / machinelearning

ML.NET is an open source and cross-platform machine learning framework for .NET.
https://dot.net/ml
MIT License
9.05k stars 1.89k forks source link

[Model management] Save/Load Model into a Relational Database (SQL Server) #4285

Open CESARDELATORRE opened 5 years ago

CESARDELATORRE commented 5 years ago

From feedback from a customer using SQL Server Functions and Stored Procedures implemented in C#:

Basically, would be good to have an API like the following:

.SaveModelToDb() .LoadModelFromDb()

The reasons and scenarios are because you run C# code only as code running within SQL Server such as a C# SQL Server Function or a Stored Procedure that is scoring an ML.NET model while doing a query or transactions.

And not just for scoring but also for saving the model after training close to the database.. and since it is a table you could also have multiple model versions..

Another scenario would be for traditional client/server apps with the client apps directly accessing a database...

Doing it that way everything would be held, and more importantly, secured in the database server without external dependencies.

ZedZipDev commented 5 years ago

Great! Yet another reason may be: A customer works with system like Client->DatabaseServer, Client->WCF/RESTFul Service->DatabaseServer but has very strong internal network policy. For example: client workstations have very restricted firewall rules for input/output ports, internet access is on some machines only and only via proxy etc. In this case it is hard to use even loading from url in internal network too. I know companies like this. They require closed environment for most of clients workstations, In this case the suitable solution may be save/load model to a databases.

markusweimer commented 5 years ago

+1

I believe saving models in databases is important for many reasons. Models represent condensed data, and therefore should be afforded the same protections as the data itself.

Regarding the implementation, we might want to look towards something like mlflow, which separates the API for storing and retrieving models from their physical location. Storing models in a database has recently been added to that feature set.

ZedZipDev commented 5 years ago

By the way I would add a note: Save/Load Model into a Relational Database (SQL Server, PostgreSQL etc) At least Azure and AWS have the "standard" set of RDBMS like this. The same as the current ML.NET preview has the feature to load training data from database and not limited to SQL Server only.

win32nipuh commented 4 years ago

Could you please estimate: Can we wait its implementation soon?

frank-dong-ms-zz commented 4 years ago

@natke @harishsk add Natalie and Harish to answer this.

Could you please estimate: Can we wait its implementation soon?

win32nipuh commented 4 years ago

It seems the Septembers Release does not contain this feature....

win32nipuh commented 3 years ago

Hi @CESARDELATORRE could you please provide your estimation for this feature implementation? Sure, very approximately but any way it is interesting. Thank you

ZedZipDev commented 2 years ago

ML.NET team what about this feature?

michaelgsharp commented 2 years ago

@luisquintanilla for prioritization. I'll add to future milestone for now since we will want this, but I'll let you figure out where it falls.

ZedZipDev commented 1 year ago

Hi Everybody, after yet one year I'd like to ask: what about this issue?