bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
https://bentoml.com
Apache License 2.0
7.13k stars 791 forks source link

Pytorch Runner fails to serve models because resource_quota is no longer used #2366

Closed timliubentoml closed 2 years ago

timliubentoml commented 2 years ago

Describe the bug

This line in the runner does not evaluate properly: https://github.com/bentoml/BentoML/blob/main/bentoml/_internal/frameworks/common/pytorch.py#L72

We then get a "set_num_threads expects a positive integer":

                         File "/opt/conda/lib/python3.9/site-packages/bentom
                       l/_internal/frameworks/common/pytorch.py", line 84,
                       in _configure
                           torch.set_num_threads(self._num_threads)
                       RuntimeError: set_num_threads expects a positive
                       integer

For the entire stack trace see: https://bentoml.slack.com/archives/CKRANBHPH/p1647887573742579?thread_ts=1647536520.735299&cid=CKRANBHPH

To Reproduce

Expected behavior

Screenshots/Logs

Environment:

Additional context

aarnphm commented 2 years ago

cc @larme

ssheng commented 2 years ago

Fixed in release 1.0.0.