Closed PonteIneptique closed 5 years ago
Hi @PonteIneptique Thanks for taking this initiative and sharing it! I am certainly interested in the statistics, so please keep me updated. :)
Hey :) Here are my first results, without touching anything of the original configuration though.
INFO:tensorflow:Finished evaluation at 2018-06-29-14:33:43
INFO - tensorflow - Finished evaluation at 2018-06-29-14:33:43
INFO:tensorflow:Saving dict for global step 2750: eval/CER = 0.035666704, eval/accuracy = 0.5859507, global_step = 2750, loss = inf
INFO - tensorflow - Saving dict for global step 2750: eval/CER = 0.035666704, eval/accuracy = 0.5859507, global_step = 2750, loss = inf
The loss=inf
seems weird to me. I sometime run into this log
2018-06-29 14:29:34.027754: W tensorflow/core/util/ctc/ctc_loss_calculator.cc:144] No valid path found.
which apparently is tied to input : https://stackoverflow.com/questions/45130184/ctc-loss-error-no-valid-path-found
For your information, my training data look like
leseuesqesâleshautespersonesdedui
and perform around 95% with Ocropus 1 ( I don't remember the exact number).
Great that you made it work!
Concerning the infinite loss it is indeed due to the tf.nn.ctc_loss
in this line.
Concerning the 95% performance, does this means that the CER is of 5% or that the WER is of 5% ?
The printed info you get for the evaluation corresponds to CER
and 1-WER
(accuracy is not a very explicit name, I may change this...)
Ocropy is not evaluating WER unfortunately ;)
One of the things that interest me in this algorithm, compared to Ocropus, is speed of training + elasticity on unseen data. I am gonna evaluate this later though :)
BTW I notice you seem to work on binarized data? You probably would get better result by using directly the images without binarization.
Yeah. Unfortunately, our training data were definitely done with these data so... Might be complicated to go back with the original ones
Maybe you know of a script which would allow me to match easily my binarized data with original ones ?
Would you mind sharing some numbers, like size of training set eval/test set and time of training ? I would be curious to know.
I don't know ocropy format at all, but if you have a file where there are the bounding boxes coordinates of the line extraction, you could go back to the original document to extract the unaltered lines.
Unfortunately for some we lost this information cleaning some lines, hence the question :) I have the numbers on my Linux partition. It's 21.40 here, I'll get back to you :)
I am currently try to match back the images with OpenCV and get non-binarized, grayscale data.
Here are the informations :
So, I have tried to rematch my binarized training data to my current binarized data but I have not enough confidence in the result (My try-outs are available here : https://github.com/PonteIneptique/template_matching/blob/master/Ocropus%20Rematch%20.ipynb ) : the process behind these training data spanned over 3 years, and the original data are in different shape than today ones, and I am pretty sure looking at the file I lost the original columns ...
So, getting back to non-binarized would be a long process that cannot be fully automated with the limited knowledge I have in CV :/
Hi @solivr ! The idea of evaluating your tool is really interesting to us. We have worked with Ocropus with medieval litterary manuscripts ( https://graal.hypotheses.org/786 ) and I am really looking forward comparing both results with handscript that are close to typescript. If you are in anyway interested, I'll post the small script I put together for converting our data for your input format ( https://github.com/PonteIneptique/ocropus-to-tf-crnn ) and keep you uptdated with the statistics :)