search

abdelaziz-mahdy / pytorch_lite

flutter package to help run pytorch lite models classification and YoloV5 and YoloV8.
MIT License
49 stars 18 forks source link

The library size is very large #57

Open denesbartha opened 8 months ago

denesbartha commented 8 months ago

I would like to use the library pytorch_lite for object detection using a YOLO8 model.

However I saw that the size of the APK file is too large, and then I found that the file libpytorch_jni.so is ~74MB. It does not matter whether I specify the version explicitly pytorch_lite: ^4.2.1 or I provide the path: ../ to the pytorch_lite package, the result is roughly the same.

Is there a way to shrink the lib size somehow by removing some unnecessary dependencies?

abdelaziz-mahdy commented 8 months ago

Sadly I couldn't find a way to do so, prs are always welcomed in this matter

noob8boi commented 6 months ago

I would like to use the library pytorch_lite for object detection using a YOLO8 model.

However I saw that the size of the APK file is too large, and then I found that the file libpytorch_jni.so is ~74MB. It does not matter whether I specify the version explicitly pytorch_lite: ^4.2.1 or I provide the path: ../ to the pytorch_lite package, the result is roughly the same.

Is there a way to shrink the lib size somehow by removing some unnecessary dependencies?

I was facing the same issue and after some research found out that yolo v8 model can be exported with --optimize flag which will call the _save_for_lite_interpreter function and now the torchscript model will work with lite dependencies. After that I uncommented the dependencies inside the build.gradle file :- 

implementation 'org.pytorch:pytorch_android_lite:2.1.0'
implementation 'org.pytorch:pytorch_android_torchvision_lite:2.1.0'

and commented these :- 

implementation 'org.pytorch:pytorch_android:2.1.0'
implementation 'org.pytorch:pytorch_android_torchvision:2.1.0'

With that you also have to use LiteModuleLoader.load instead of Module.load for loading your model in PytorchLitePlugin.java file & import org.pytorch.LiteModuleLoader in the same file.

abdelaziz-mahdy commented 6 months ago

When I tried that the size didn't change much in my case , can you let me know the difference in size between the normal one and lite one

And if you don't mind can you open a pr?

noob8boi commented 6 months ago

When I tried that the size didn't change much in my case , can you let me know the difference in size between the normal one and lite one

And if you don't mind can you open a pr?

earlier libpytorch_jni.so was around 70MB but now 18.7MB.. was it ~70MB for you before & after the change? image

Regarding the PR.. should I make the code changes as mentioned above?

abdelaziz-mahdy commented 6 months ago

When I tried that the size didn't change much in my case , can you let me know the difference in size between the normal one and lite one

And if you don't mind can you open a pr?

earlier libpytorch_jni.so was around 70MB but now 18.7MB.. was it ~70MB for you before & after the change? image

Regarding the PR.. should I make the code changes as mentioned above?

Yes for the pr same changes you mentioned, and I meant the size of the apk using normal and _lite

noob8boi commented 6 months ago

When I tried that the size didn't change much in my case , can you let me know the difference in size between the normal one and lite one And if you don't mind can you open a pr?

earlier libpytorch_jni.so was around 70MB but now 18.7MB.. was it ~70MB for you before & after the change? image Regarding the PR.. should I make the code changes as mentioned above?

Yes for the pr same changes you mentioned, and I meant the size of the apk using normal and _lite

image

i recorded this old build yesterday but for some reason i am unable to reproduce the 330MB build again even after deleting pubspec.lock , clearing all cache etc. Now I keep getting 130MB builds with/without the above changes. Maybe the code change isn't making much of a difference after all 🤔 .. I will reply as soon as I figure this out.

noob8boi commented 6 months ago

Okay so flutter build apk --target-platform android-arm64 --analyze-size outputs two APKs and 18MB is gone with the lite dependency. Still not sure why I was getting 330MB yesterday. Anyway gonna create a PR now 👍

image image

noob8boi commented 6 months ago

https://github.com/zezo357/pytorch_lite/pull/68

abdelaziz-mahdy commented 6 months ago

Thank you very much, will run workflows and test if everything good

abdelaziz-mahdy commented 6 months ago

as i mentioned the difference is not much 5mb you pr https://github.com/zezo357/pytorch_lite/actions/runs/7348278520/job/20007199904

image

before it https://github.com/zezo357/pytorch_lite/actions/runs/7185854080/job/19569974871

image

the problem is i dont know if the reduction in size hurts the models options that can be ran? and the reduction is roughly 5mb i dont know if it worth taking the risk

confirm with me if you any other opinion i think it can be published as a work in progress (beta) while we convert to pytorch lite and latest ios version if its stable i think it has some problems https://github.com/zezo357/pytorch_lite/issues/63

noob8boi commented 6 months ago

as i mentioned the difference is not much 5mb you pr https://github.com/zezo357/pytorch_lite/actions/runs/7348278520/job/20007199904 image

before it https://github.com/zezo357/pytorch_lite/actions/runs/7185854080/job/19569974871 image

the problem is i dont know if the reduction in size hurts the models options that can be ran? and the reduction is roughly 5mb i dont know if it worth taking the risk

confirm with me if you any other opinion i think it can be published as a work in progress (beta) while we convert to pytorch lite and latest ios version if its stable i think it has some problems #63

I understand.. actually i posted the comment because i thought this code change will help in reducing libpytorch_jni.so file from 70MB to less than 15MB (libpytorch_jni_lite.so) and solve OP's problem. But I have no clue what was the actual cause of it.

I guess the issue can remain open until further optimizations are found then we can consider creating another PR

noob8boi commented 6 months ago

if you don't mind can you build output apk file with --analyze-size flag. I want to see the space taken by assets and the libpytorch_jni_lite.so

abdelaziz-mahdy commented 6 months ago

if you don't mind can you build output apk file with --analyze-size flag. I want to see the space taken by assets and the libpytorch_jni_lite.so

Sadly I can't do it now, maybe tomorrow and I will let you know

abdelaziz-mahdy commented 6 months ago

added it in master and in your pr

abdelaziz-mahdy commented 6 months ago

pr: https://github.com/zezo357/pytorch_lite/actions/runs/7351145699/job/20014024960

✓  Built build/app/outputs/flutter-apk/app-release.apk (157.1MB).
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
app-release.apk (total compressed)                                        157 MB
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  META-INF/
    CERT.SF                                                                 7 KB
    CERT.RSA                                                              1017 B
    MANIFEST.MF                                                             6 KB
  assets/
    flutter_assets                                                         83 MB
  classes.dex                                                               3 MB
  lib/
    arm64-v8a                                                              15 MB
    armeabi-v7a                                                            21 MB
    Dart AOT symbols accounted decompressed size                            5 MB
      package:flutter                                                       2 MB
      package:image                                                      1023 KB
      dart:core                                                           335 KB
      dart:typed_data                                                     277 KB
      dart:ui                                                             259 KB
      dart:async                                                          108 KB
      dart:collection                                                      97 KB
      dart:convert                                                         71 KB
      dart:ffi                                                             59 KB
      dart:io                                                              54 KB
      package:pytorch_lite_example                                         32 KB
      package:vector_math/
        vector_math_64.dart                                                32 KB
      dart:isolate                                                         31 KB
      package:pytorch_lite                                                 30 KB
      package:camera_platform_interface                                    25 KB
      package:archive                                                      21 KB
      package:path                                                         16 KB
      package:camera_android/
        src                                                                14 KB
      package:camera/
        src                                                                13 KB
      package:image_picker_android                                         10 KB
    x86                                                                    18 MB
    x86_64                                                                 17 MB
  LICENSE-junit.txt                                                         4 KB
  junit/
    runner                                                                  2 KB
  kotlin/
    collections                                                             1 KB
    kotlin.kotlin_builtins                                                  5 KB
    ranges                                                                  1 KB
    reflect                                                                 1 KB
  AndroidManifest.xml                                                       2 KB
  res/
    CG.png                                                                  1 KB
    _W.png                                                                  1 KB
    yq.png                                                                  1 KB
  resources.arsc                                                           [53](https://github.com/zezo357/pytorch_lite/actions/runs/7351145699/job/20014024960#step:6:54) KB
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒

master: https://github.com/zezo357/pytorch_lite/actions/runs/7351120618/job/20013953038

Running Gradle task 'assembleRelease'...                          315.4s
✓  Built build/app/outputs/flutter-apk/app-release.apk (175.1MB).
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
app-release.apk (total compressed)                                        175 MB
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  META-INF/
    CERT.SF                                                                 7 KB
    CERT.RSA                                                              1017 B
    MANIFEST.MF                                                             6 KB
  assets/
    flutter_assets                                                         83 MB
  classes.dex                                                               3 MB
  lib/
    arm64-v8a                                                              19 MB
    armeabi-v7a                                                            25 MB
    Dart AOT symbols accounted decompressed size                            5 MB
      package:flutter                                                       2 MB
      package:image                                                      1023 KB
      dart:core                                                           335 KB
      dart:typed_data                                                     277 KB
      dart:ui                                                             259 KB
      dart:async                                                          108 KB
      dart:collection                                                      97 KB
      dart:convert                                                         71 KB
      dart:ffi                                                             59 KB
      dart:io                                                              54 KB
      package:pytorch_lite_example                                         32 KB
      package:vector_math/
        vector_math_64.dart                                                32 KB
      dart:isolate                                                         31 KB
      package:pytorch_lite                                                 30 KB
      package:camera_platform_interface                                    25 KB
      package:archive                                                      21 KB
      package:path                                                         16 KB
      package:camera_android/
        src                                                                14 KB
      package:camera/
        src                                                                13 KB
      package:image_picker_android                                         10 KB
    x86                                                                    22 MB
    x86_64                                                                 22 MB
  LICENSE-junit.txt                                                         4 KB
  junit/
    runner                                                                  2 KB
  kotlin/
    collections                                                             1 KB
    kotlin.kotlin_builtins                                                  5 KB
    ranges                                                                  1 KB
    reflect                                                                 1 KB
  AndroidManifest.xml                                                       2 KB
  res/
    CG.png                                                                  1 KB
    _W.png                                                                  1 KB
    yq.png                                                                  1 KB
  resources.arsc                                                           [53](https://github.com/zezo357/pytorch_lite/actions/runs/7351120618/job/20013953038#step:6:54) KB
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
A summary of your APK analysis can be found at: /Users/runner/.flutter-devtools/apk-code-size-analysis_01.json
noob8boi commented 6 months ago

▒▒▒▒▒▒▒

As you can see 83MB is taken by the flutter assets which I believe is from the model. So I don't think the package can perform any further optimizations over there. And armabi-v7a file is only around 21MB.. so the libpytorch_jni_lite.so is definitely not 70MB .. more like 15MB or so. The analysis json file should be able give in depth view with the help of dart dev tools 🤔.. I hope I'm not drawing incorrect conclusions. Basically I don't think there can be major reduction in sizes now. Best we can do is reduce the model size but that's not something this package is made for.

abdelaziz-mahdy commented 6 months ago

▒▒▒▒▒▒▒

As you can see 83MB is taken by the flutter assets which I believe is from the model. So I don't think the package can perform any further optimizations over there. And armabi-v7a file is only around 25MB.. so the libpytorch_jni_lite.so is definitely not 70MB .. more like 15MB or so. The analysis json file should be able give in depth view with the help of dart dev tools 🤔.. I hope I'm not drawing incorrect conclusions. Basically I don't think there can be major reduction in sizes now. Best we can do is reduce the model size but that's not something this package is made for.

Yes I agree the power is in pytorch themselves, for the problem of assets I will try to split app and models so I download models at run time to show true app size

Sadly I can't do it now and don't think I will be able for the next days

Anyway thank you very much for your contribution and for checking the possibility of reducing it to that depth, hopefully pytorch reduces their lib size

noob8boi commented 6 months ago

▒▒▒▒▒▒▒

As you can see 83MB is taken by the flutter assets which I believe is from the model. So I don't think the package can perform any further optimizations over there. And armabi-v7a file is only around 25MB.. so the libpytorch_jni_lite.so is definitely not 70MB .. more like 15MB or so. The analysis json file should be able give in depth view with the help of dart dev tools 🤔.. I hope I'm not drawing incorrect conclusions. Basically I don't think there can be major reduction in sizes now. Best we can do is reduce the model size but that's not something this package is made for.

Yes I agree the power is in pytorch themselves, for the problem of assets I will try to split app and models so I download models at run time to show true app size

Sadly I can't do it now and don't think I will be able for the next days

Anyway thank you very much for your contribution and for checking the possibility of reducing it to that depth, hopefully pytorch reduces their lib size

happy to help :)

abdelaziz-mahdy commented 6 months ago

Btw I will leave your pr open, since it's valid until the assets stuff is handled if it's better then I guess we have a solution