PytorchModelArtifact vs PytorchLightningModelArtifact question

[HansolEom]

I use PytorchLightningModelArtifact. I get this problem:

Python builtin <built-in method apply of FunctionMeta object at 0x7fea84545080> is currently not supported in Torchscript:
  File "/opt/anaconda3/envs/bento/lib/python3.7/site-packages/efficientnet_pytorch/utils.py", line 80
    def forward(self, x):
        return SwishImplementation.apply(x)
               ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE

when doing model.pack. If i use the torch model in pytorch lightning, this problem does not occur. For what reason?

error code:

from PIL import Image

import torch
from torchvision import transforms

import bentoml
from bentoml.artifact import PytorchLightningModelArtifact
from bentoml.handlers import ImageHandler

transform = transforms.Compose(
    [
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
    ]
)

# we will use cpu for prediction
cpu = torch.device("cpu")

# Defines the artifact that is used to deserialize the model
@bentoml.artifacts([PytorchLightningModelArtifact("model")])
class AntOrBeeClassifier(bentoml.BentoService):
    @bentoml.api(ImageHandler)
    def predict(self, img):

        img = Image.fromarray(img)
        img = transform(img)

        self.artifacts.model.eval()
        outputs = self.artifacts.model(img.unsqueeze(0))

        _, idxs = outputs.topk(1)
        idx = idxs.squeeze().item()
        return classes[idx]

def saveToBento():

    model_ft = custumodel # pytorchlightning

    bento_svc = AntOrBeeClassifier()
    bento_svc.pack("model", model_ft)

    # Save Bento Service
    saved_path = bento_svc.save()
    print("Bento Service Saved in ", saved_path)

if __name__ == "__main__":
    saveToBento()

torchmodel code:

from PIL import Image

import torch
from torchvision import transforms

import bentoml
from bentoml.artifact import PytorchModelArtifact
from bentoml.handlers import ImageHandler

transform = transforms.Compose(
    [
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
    ]
)

# we will use cpu for prediction
cpu = torch.device("cpu")

# Defines the artifact that is used to deserialize the model
@bentoml.artifacts([PytorchModelArtifact, ("model")])
class AntOrBeeClassifier(bentoml.BentoService):
    @bentoml.api(ImageHandler)
    def predict(self, img):

        img = Image.fromarray(img)
        img = transform(img)

        self.artifacts.model.eval()
        outputs = self.artifacts.model(img.unsqueeze(0))

        _, idxs = outputs.topk(1)
        idx = idxs.squeeze().item()
        return classes[idx]

def saveToBento():

    model_ft = custumodel.model # pytorchlightning

    bento_svc = AntOrBeeClassifier()
    bento_svc.pack("model", model_ft)

    # Save Bento Service
    saved_path = bento_svc.save()
    print("Bento Service Saved in ", saved_path)

if __name__ == "__main__":
    saveToBento()

my env: bentoml = 013.1 pytorch = 1.10.1 pytorch-lightning = 1.5.7

[ChoYongchae]

Can you check if member function set_swish() is in self.artifacts.model of your code? (I'm newbie in BentoML, and I've only used pytorch)

From your error message, Bento seems to be using EfficientNet in pytorch. _(File "/opt/anaconda3/envs/bento/lib/python3.7/site-packages/efficientnetpytorch/) For export this model, you should set_swish(memory_efficient=False).

You can refer "https://github.com/lukemelas/EfficientNet-PyTorch/blob/master/efficientnet_pytorch/model.py", _def set_swish(self, memory_efficient=True): """Sets swish function as memory efficient (for training) or standard (for export). Args: memory_efficient (bool): Whether to use memory-efficient version of swish. """ self._swish = MemoryEfficientSwish() if memoryefficient else Swish()

[aarnphm]

This doesn't seem like a BentoML issue. You can try out the recent rc releases as we changed the API quite a bit.

Install the latest rc releases with pip install --pre -U bentoml

Feel free to open this again. Closing for now.