bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
https://bentoml.com
Apache License 2.0
7.15k stars 792 forks source link

bug: Compiled functions can't take variable number of arguments or use keyword-only arguments with defaults #4333

Closed holzweber closed 9 months ago

holzweber commented 11 months ago

Describe the bug

I try to use BentoML for a Lightning Model. The model is a pretrained network from the anomalib library (EfficientAD) https://github.com/openvinotoolkit/anomalib/tree/main/src/anomalib/models/efficient_ad

When I try to save the model with bentoml.pytorch_lightning.save_model, i get the following error:

NotSupportedError: Compiled functions can't take variable number of arguments or use keyword-only arguments with defaults: File "C:\~\lib\site-packages\anomalib\models\components\base\anomaly_module.py", line 56 def forward(self, batch: dict[str, str | Tensor], *args, **kwargs) -> Any:


        """Forward-pass input tensor to the module.

As I do not want to change anything in the Anomalib library, is there a known solution?

### To reproduce

1. pip install anomalib
2. pip install bentoml

from anomalib.models.efficient_ad.lightning_model import EfficientAd
import bentoml

model = EfficientAd(384, [100,100])
bentoml.pytorch_lightning.save_model("efficient_ad", model)

### Expected behavior

_No response_

### Environment

anomalib: 0.7.0
bentoml: 1.1.10
pytorch_lightning: 1.7.0
holzweber commented 9 months ago

Workaround: store lightning model as torch or onnx