search

Kotlin / kotlindl

High-level Deep Learning Framework written in Kotlin and inspired by Keras
Apache License 2.0
1.47k stars 103 forks source link

[FeatureRequest] Add EAST text detection support #336

Open rcd27 opened 2 years ago

rcd27 commented 2 years ago

There is very usefull pretrained Tensorflow model for detecting text on image: https://github.com/argman/EAST

Would be nice to use that in KotlinDL

Opencv.DNN module has such a support: python tutorial OpenCV doc reference I've also ported it to Kotlin, works fine (a lot of boilerplate, but OK)

rcd27 commented 2 years ago

This is a Kotlin implementation with use of OpenCV.DNN module + EAST detection model: https://github.com/rcd27/opencv-EAST-kotlin

zaleslaw commented 2 years ago

It's an interesting proposal. A have a few questions before we move forward. Is the model saved to .pb format generated by Tensorflow version 1.x? Looks like yes, but we should now. If yes, I suggest implementing an example using, ONNX module for example or Tensorflow Saved Module + Image DSL preprocessing.

The best way to convert to ONNX via https://github.com/onnx/tensorflow-onnx project in Colab, for example.

If you need some help on all steps, please let me know

rcd27 commented 2 years ago

Yes, the model format is .pb, I'll try to go through the steps you suggested.

rcd27 commented 2 years ago

@zaleslaw all EAST models with .pb format I found in the internet were missing tag value (checked with saved_model_cli tool). That makes tensorflow-onnx to throw error (missing tag) while trying to convert to ONNX format. Any ideas how to workaround this?

zaleslaw commented 2 years ago

Looks like it requires a tag setting before convertation I found a few contants here https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/saved_model/tag_constants.py#L27