search

pytorch / serve

Serve, optimize and scale PyTorch models in production
https://pytorch.org/serve/
Apache License 2.0
4.21k stars 859 forks source link

Allow .tar.gz models to be loaded using load_models=all #3320

Closed m10an closed 1 month ago

m10an commented 1 month ago

🚀 The feature

Allow to load on server startup .tar.gz model archives using property load_models=all, besides .mar and .model files

Motivation, pitch

I am currently using TorchServe to serve models archived with the --archive-format=tgz flag. The model storage folder contains multiple versions of the same model, but currently, it is impossible to load all of them on server startup because it only works with .mar and .model files.

Alternatives

No response

Additional context

I already tested simple change in code, works as expected

mreso commented 1 month ago

Hi @m10an thanks for reporting this issue. That change would be very valuable for our users I think. I am also not sure if no_archive models i.e. folder are currently included in load_models=all. Would you be interested to contribute a PR for this change? I would be happy to review it!

m10an commented 1 month ago

Hi @mreso, thank you for quick response. I will gladly contribute