search

google-ai-edge / ai-edge-torch

Supporting PyTorch models with the Google AI Edge TFLite runtime.
Apache License 2.0
364 stars 51 forks source link

Pytorch model converted to tflite is significantly slower #211

Closed lusinlu closed 1 month ago

lusinlu commented 2 months ago

Description of the bug:

Hi, we are trying to run pytorch model on android and following examples of image segmentation from google_ai_edge. We noticed that when running with the default model in the example, which is deeplab v3 the inference is ~40ms, but when we are using pytorch model (with ~2mln parameters, which is significantly smaller than deeplab) converted to tflite, the inference is ~200ms. For converting the model, we are using the simple code:

model = ModelSparce()
model.load_state_dict(torch.load(args.weights)['model'], strict=False)
model.eval()

batch = (torch.randn(1, 3, 288, 288),)

edge_model = ai_edge_torch.convert(model, batch)
edge_model.export("model_aiedge.tflite")

What possibly can be the issue here.

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

pkgoogle commented 2 months ago

Hi @lusinlu, thanks for reporting the issue. Can you let us know what hardware/environment you are using? Such as are you using a real device or an emulator? For either case, can you give us the specifications, for example: Pixel 8 Pro API 34 (Android 14.0 "UpsideDownCake") | arm 64. The more information, the better. Thanks.

github-actions[bot] commented 1 month ago

Marking this issue as stale since it has been open for 7 days with no activity. This issue will be closed if no further activity occurs.

github-actions[bot] commented 1 month ago

This issue was closed because it has been inactive for 14 days. Please post a new issue if you need further assistance. Thanks!