Text Characters appear jumbled up after detection

[silvererudite]

Hi...@fg-mindee , @AbdelfattahSekak, @charlesmindee ...I'm trying to use this demo on sample restaurant menus to extract texts however a lot of the words detected are showing up jumbled like this As you can see, the word Starters is showing up as Starierst and a lot of words are also not being recognized. Interestingly this jumbling up doesn't happen when I run the same example on the Tensorflow version deployed on Hugging Face. Is there any special constraints/ limitations for the Tensorflow JS model or am I missing something important here?

[fg-mindee]

Hi @silvererudite :wave:

Thanks for the feedback! Glad you've been able to try both our demos the TFJS is still in beta, we're working on it this very day to try to bridge the gap between both demos, but here is a short explanation of the difference:

• the demo on HF Spaces is run in pure Python (with TensorFlow). The browser is showing the results but everything is computed just like any other Python program
• the TFJS demo is different in that regard: everything is done in the browser. The computation are done with TensorFlow JS (no python at all), this comes at a cost of higher compatibility constraint (some of our models are not directly compatible with that). For this reason, we've used lighter model in TFJS
• that being said, there is a small bug being fixed in https://github.com/mindee/doctr-tfjs-demo/pull/33 by @charlesmindee. This was preventing decent performances typically on your type of documents.

So my suggestion is:

• wait for the PR to be merged
• assess whether the performances are OK for you only then :+1:
• we're gonna add a better sorting of word predictions shortly to tackle your second point (@AbdelfattahSekak has already been working on it)

Hope this helps!

[silvererudite]

@fg-mindee thanks a lot for the heads-up...will definitely try again after the fix

[charlesmindee]

This should be fixed by #33, thanks!