search

bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
https://bentoml.com
Apache License 2.0
7.13k stars 791 forks source link

feat: `bentoml.torchhub` #2704

Closed aarnphm closed 2 years ago

aarnphm commented 2 years ago

Currently there aren't a way to fully support torch hub. https://github.com/bentoml/BentoML/issues/2602 like this often comes up due to different torch hub imports implementation.

Proposal

Provides a bentoml.torchhub that create interaction between bentoml and torch hub modules.

bentoml.torchhub.import_model

bentoml.torchhub.get

For runner, to_runner() will return a PyTorchRunner

runner = bentoml.torchhub.get("pytorch_vision:latest").to_runner() # -> PyTorchRunner
parano commented 2 years ago

Should we make it just bentoml.pytorch.import_from_hub so that it's easier for PyTorch/BentoML users to discover? also since they are gonna share the same runner implementation

aarnphm commented 2 years ago

then we should also merge bentoml.torchscript to bentoml.pytorch.save_model(..., save_as_torchscript=True). Since the runner implementation are also the same.

aarnphm commented 2 years ago

I think this is a pretty low triage.