search

lukas-blecher / LaTeX-OCR

pix2tex: Using a ViT to convert images of equations into LaTeX code.
https://lukas-blecher.github.io/LaTeX-OCR/
MIT License
12k stars 984 forks source link

Confidence level of a prediction #235

Closed kdiogenes closed 1 year ago

kdiogenes commented 1 year ago

Hi,

Is it possible to know the confidence level of a prediction? I looked at the code but couldn't spot if this was possible.

I'm using LaTeX-OCR with some documents that have images that I don't know are equations or not. If it was possible to generate a confidence level of the conversion I could set a threshold to consider the image an equation or not.

Do you have any insight about this, @lukas-blecher?

Thanks in advance and congrats on the great work!

lukas-blecher commented 1 year ago

Thanks!

You could get a score during generation by multiplying the softmax activated logits / probabilites for each token in the sequence. However I don't think you can really use that number for much. In my experience these tend to be rather high regardless of the quality of the output.

If it's the equation detection you want, maybe look into an equation detection model like https://gitlab.com/dprl/scaled-yolov4