search

kha-white / manga-ocr

Optical character recognition for Japanese text, with the main focus being Japanese manga
Apache License 2.0
1.63k stars 84 forks source link

Is there a way to optimize transfomers backend binary size? #37

Open yupm opened 1 year ago

yupm commented 1 year ago

Hello.. Currently I am trying to bundle a deb package for Poricom which uses manga_ocr. However, the torch dependencies of the transformers backend are making the deb excessively large.

$ find . -type f -size +50M -exec du -sh {} \;

318M    ./opt/poricom/libcublasLt.so.11
145M    ./opt/poricom/libcublas.so.11
179M    ./opt/poricom/unidic_lite/dicdir/sys.dic
69M     ./opt/poricom/unidic_lite/dicdir/matrix.bin
438M    ./opt/poricom/libcudnn_cnn_infer.so.8
88M     ./opt/poricom/libcudnn_ops_infer.so.8
57M     ./opt/poricom/assets/languages/chi_tra.traineddata
526M    ./opt/poricom/torch/lib/libtorch_cpu.so
253M    ./opt/poricom/torch/lib/libtorch_cuda_cpp.so
728M    ./opt/poricom/torch/lib/libtorch_cuda_cu.so

I wonder if you could advise or help regarding reducing the size of the bundle by using another backend such as TF or JAX.

kha-white commented 1 year ago

It's not a priority for me right now, but I agree that it would be nice to drop the torch dependency. Another option is ONNX.

HighLiuk commented 3 weeks ago

Hey there @yupm I'm currently trying to replicate MangaOcr outside of python so I can give you some hints. You can find here #45 a way to export an ONNX model and drop both the transformers & torch dependencies. However consider that this model alone (ONNX file) is more than 500M and I guess you cannot lower that.