search

tobegit3hub / simple_tensorflow_serving

Generic and easy-to-use serving service for machine learning models
https://stfs.readthedocs.io
Apache License 2.0
757 stars 195 forks source link

Loading and Unloading versions of model #75

Open dharmendrach opened 4 years ago

dharmendrach commented 4 years ago

Usually systems which are in production require, loading of new versions on run-time and even unloading previous versions of model. Is there any way to achieve same using this framework ? I have seen tensorflow serving does this via constantly checking file system for new model versions and unloading deleted models.

I really like idea of single framework for hosting different framework models. If you have some tasks for future releases or any contribution required, I can help.

marooned20 commented 3 years ago

@drc10723 I was also hoping to find a way to handle hot-reload of the server based on the changes in the config file! Have you solved it? if so, I would be curious to learn as well.

tobegit3hub commented 3 years ago

We can not hot-reload models with config file or using non-tensorfllow models now. We may use load balancer or AB testing tools for this scenario.