search

smaranjitghose / DocLense

An open-source document scanner!
https://doclense.vercel.app/
Creative Commons Zero v1.0 Universal
149 stars 130 forks source link

OCR feature #26

Closed Rishabhc711 closed 3 years ago

Rishabhc711 commented 3 years ago

Since we are already scanning the document , it would be even better if we give the users the option of converting it into a text file or pdf once if recognises all the text characters of the scanned image I would like to work on the issue

smaranjitghose commented 3 years ago

How do you propose we intergrate the OCR model ( I do not recommend using Tesseract) into the Flutter APP?

Rishabhc711 commented 3 years ago

I am not very sure of the exact integration method we could use for it. But we can use a pre-trained model's weights, train it on our use case's dataset and then we can look forward to integrating it into the app.

As far as i know we can plugin our own tensorflow models in the flutter app

Rishabhc711 commented 3 years ago

https://medium.com/@am15hg/text-classification-using-tensorflow-lite-plugin-for-flutter-3b92f6655982

This article deals with the problem, but it doesn't train its own model , instead its directly integrates a model from tensorflow.

smaranjitghose commented 3 years ago

I am not very sure of the exact integration method we could use for it. But we can use a pre-trained model's weights, train it on our use case's dataset and then we can look forward to integrating it into the app.

As far as i know we can plugin our own tensorflow models in the flutter app

Yes I am aware of using tflite with Flutter. But for that you need to train your own model, quantize it and then convert it to tflite

Rishabhc711 commented 3 years ago

I think I can use the pretrained weights to create a new model, train it on a dataset on GPU-enabled google colab. Since it is fort he purpose of OCR, i think it can be done. I have done a similar docscanner project in the past but we didnt worry for its integration with a mobile app back then. Seems challenging , but worth a try! What do you say @smaranjitghose?

Rishabhc711 commented 3 years ago

Should I work on this issue? @smaranjitghose

Rishabhc711 commented 3 years ago

And can u add the DWOC label if I proceed with this? @smaranjitghose

smaranjitghose commented 3 years ago

I think I can use the pretrained weights to create a new model, train it on a dataset on GPU-enabled google colab. Since it is fort he purpose of OCR, i think it can be done. I have done a similar docscanner project in the past but we didnt worry for its integration with a mobile app back then. Seems challenging , but worth a try! What do you say @smaranjitghose?

Unless we can integrate the model, the entire effort would be futile. Read some blogs on it and get back to us