search

ChaoningZhang / MobileSAM

This is the official code for MobileSAM project that makes SAM lightweight for mobile applications and beyond!
Apache License 2.0
4.79k stars 500 forks source link

Getting mobileSAM into a format that runs on mobile (tflite, torchscript) #141

Open cummins-orgs opened 8 months ago

cummins-orgs commented 8 months ago

Hi mobileSAM creators!

Thanks for the project, it's really awesome.

Maybe this is my stupidity, but I thought I should highlight this: I've essentially tried and failed over a couple of days to get mobileSAM into either torchscript or tflite. Possible that direct conversion to e.g. CoreML would be painless, haven't tried that just yet.

A lot of this isn't traceable to you: converting SAM for use on mobile appears to be non-trivial.

But: one suggestion for the future, if you can spare the time: offer links/walkthrough for getting this project in shape for use in mobile apps.

Or, if that exists, I'd love to follow it..!

Thanks again,

Matt

SkyloveQiu commented 7 months ago

Did you make any progress, maybe we can cooperate.

PierreMardon commented 4 months ago

Hi @cummins-orgs @SkyloveQiu ! If you had a look at EdgeSAM, they implemented a CoreML output. I implemented an integration (closed source for now) of it. The thing is that their license is a lot less permissive.

I'm interested by the CoreML output of MobileSAM as well, but did not investigate yet. Don't hesitate to reach out if you're still interested / already made any progress ;)

cummins-orgs commented 4 months ago

Hi both,

After a lot of trouble the project I was working on got cancelled. Maybe coreml was the way to go! The issues I had in swift were either to do with the actual model file or to do with pre-processing fwiw. Former is much more complex/verbose in swift than python

On Thu, Jun 27, 2024 at 8:54 AM, Pierre Mardon @.***(mailto:On Thu, Jun 27, 2024 at 8:54 AM, Pierre Mardon < wrote:

Hi @.(https://github.com/cummins-orgs) @.(https://github.com/SkyloveQiu) ! If you had a look at EdgeSAM, they implemented a CoreML output. I implemented an integration (closed source for now) of it. The thing is that their license is a lot less permissive.

I'm interested by the CoreML output of MobileSAM as well, but did not investigate yet. Don't hesitate to reach out if you're still interested / already made any progress ;)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

PierreMardon commented 4 months ago

Ok thanks @cummins-orgs ! I think indeed CoreML is the preferred / simplest path ;)

kestraltorah commented 2 months ago

我已经实现了移动端的部署，并且在我的软件中已经应用一年多了

PierreMardon commented 2 months ago

Hi @kestraltorah ! Could you provide more insight on how you performed your integration please?

kestraltorah commented 2 months ago

使用onnx，编码模型+解码模型

---- Replied Message ---- | From | Pierre @.> | | Date | 08/21/2024 00:11 | | To | ChaoningZhang/MobileSAM @.> | | Cc | kestraltorah @.>, Mention @.> | | Subject | Re: [ChaoningZhang/MobileSAM] Getting mobileSAM into a format that runs on mobile (tflite, torchscript) (Issue #141) |

Hi @kestraltorah ! Could you provide more insight on how you performed your integration please?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

PierreMardon commented 2 months ago

Oh right, I didn't even consider it was possible to just use ONNX directly in iOS 🤷‍♂️ I guess I'll try that, thx !