search

mittagessen / kraken

OCR engine for all the languages
http://kraken.re
Apache License 2.0
750 stars 131 forks source link

Optimized distribution of packet for inference only #614

Closed dantetemplar closed 5 months ago

dantetemplar commented 5 months ago

I think it would be great to be able to quickly and easily add kraken to a project. But this is hampered by the installation of large dependency packages: torch, cuda stuff and others. Perhaps using ONNX would help a lot.

mittagessen commented 5 months ago

ONNX doesn't play well with variable shape tensors as are needed for our text recognition models. Nor do any of the other graph compilation/capture approaches for pytorch (torchscript, coreml, ...) so there isn't currently a technically feasible way to run lightweight deployments.

dantetemplar commented 5 months ago

ONNX doesn't play well with variable shape tensors as are needed for our text recognition models. Nor do any of the other graph compilation/capture approaches for pytorch (torchscript, coreml, ...) so there isn't currently a technically feasible way to run lightweight deployments.

Thanks for the answer, I understood regarding ONNX. Do you know any alternatives?

mittagessen commented 5 months ago

On 24/06/17 10:18AM, dantetemplar wrote:

Thanks for the answer, I understood regarding ONNX. Do you know any alternatives?

All of those deployment frameworks suffer from it to some extent as far as I know. In theory you can often construct the models manually with their primitives, e.g. in CoreML, but this is tedious and often they behave differently from native pytorch layers or are missing layer types so it is rarely a simple conversion. Compilers using introspection usually blow up with variable size tensors.