How to deploy ensemble models of different versions more elegantly?

[lzcchl]

Currently, I already know how to deploy different versions of an ordinary model. Suppose we now have two models, namely resnet_v1.pth and resnet_v2.pth. Then, in my model_repo/resnet_pytorch directory, there will be folders "1" and "2", where these two model files are placed respectively. Then, in config.pbtxt, only by setting the parameter "version_policy" to "{ all{} }", these two models can be deployed simultaneously. The deployment log is as follows (Now, there is only one folder in model_repo, named "resnet_pytorch"): +------------------------------------------------------+---------+--------+ | Model | Version | Status | +------------------------------------------------------+---------+--------+ | resnet_pytorch | 1 | READY | | resnet_pytorch | 2 | READY | +------------------------------------------------------+---------+--------+

However, in actual use, we often use ensemble models, such as preprocess+resnet_v1.pth and preprocess+resnet_v2.pth. Currently, my approach is to create two folders under model_repo, called ensemble_resnet_pytorch_v1 and ensemble_resnet_pytorch_v2 respectively. In the two config.pbtxt files, I will set the model_version under ensemble_scheduling to 1 and 2 respectively. The deployment log is as follows (Now, there are four folder in model_repo, named "resnet_pytorch", "resnet_preprocess", "ensemble_resnet_pytorch_v1", "ensemble_resnet_pytorch_v2"): +------------------------------------------------------+---------+--------+ | Model | Version | Status | +------------------------------------------------------+---------+--------+ | ensemble_resnet_pytorch_v1 | 1 | READY | | ensemble_resnet_pytorch_v2 | 1 | READY | | resnet_preprocess | 1 | READY | | resnet_pytorch | 1 | READY | | resnet_pytorch | 2 | READY | +------------------------------------------------------+---------+--------+

But, I think creating two folders("ensemble_resnet_pytorch_v1", "ensemble_resnet_pytorch_v2") for different ensemble models respectively is not elegant enough. Is there a more appropriate way to deploy ensemble models of different versions (only one folder "ensemble_resnet_pytorch" with two version)?

[dong-toggle-ai]

same here, would be good to know the best way to use multiple versions for ensemble models. have you got any idea yet?