search

goncalopp / simple-ocr-opencv

A simple python OCR engine using opencv
GNU Affero General Public License v3.0
524 stars 175 forks source link

Changing the training image & Creating grounding files #10

Closed richard512 closed 8 years ago

richard512 commented 8 years ago

Step 1) Tried to change the training image in example.py

ocr.train( ImageFile('otherimagefile') )

Step 2) Bumped into an error

Exception: The provided file is not grounded

Step 3) Tried the solution issue 2 suggests

from files import ImageFile
from grounding import UserGrounder
from segmentation import ContourSegmenter, draw_segments
from feature_extraction import SimpleFeatureExtractor
from classification import KNNClassifier
from ocr import OCR, accuracy, show_differences, reconstruct_chars

segmenter=  ContourSegmenter( blur_y=5, blur_x=5, block_size=11, c=10)
extractor=  SimpleFeatureExtractor( feature_size=10, stretch=False )
classifier= KNNClassifier()
ocr= OCR( segmenter, extractor, classifier )

test_image= ImageFile('cap1')
test_classes, test_segments= ocr.ocr( test_image, show_steps=True )
grounder= UserGrounder()
grounder.ground(test_image, test_segments)
test_image.ground.write()

Step 4) But an error occurred:

OpenCV Error: Assertion failed (N >= K) in kmeans, file /build/buildd/opencv-2.4.8+dfsg1/modules/core/src/matrix.cpp, line 2702
Traceback (most recent call last):
  File "example.py", line 14, in <module>
    test_classes, test_segments= ocr.ocr( test_image, show_steps=True )
  File "/home/username/simple-ocr-opencv/ocr.py", line 40, in ocr
    segments= self.segmenter.process( image_file.image )
  File "/home/username/simple-ocr-opencv/processor.py", line 87, in process
    output= self._process(arguments)
  File "/home/username/simple-ocr-opencv/processor.py", line 131, in _process
    arguments= p.process( arguments )
  File "/home/username/simple-ocr-opencv/processor.py", line 87, in process
    output= self._process(arguments)
  File "/home/username/simple-ocr-opencv/segmentation_aux.py", line 63, in _process
    tops=               self._guess_lines( segment_tops )
  File "/home/username/simple-ocr-opencv/segmentation_aux.py", line 29, in _guess_lines
    compactness, classified_points, means = cv2.kmeans( data=ys, K=k, bestLabels=None, criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_MAX_ITER, 1, 10), attempts=2, flags=cv2.KMEANS_PP_CENTERS)
cv2.error: /build/buildd/opencv-2.4.8+dfsg1/modules/core/src/matrix.cpp:2702: error: (-215) N >= K in function kmeans
goncalopp commented 8 years ago

You're in for some debugging :) I'll assume you've tried to ground the provided image files and that is working.

N >= K in function kmeans: K-means is a clustering algorithm. It's used to guess the lines of text of the image. K, by convention, is the number of clusters, and N is probably the number of samples. I'm guessing what happened is the number of samples is too low (ys variable).

https://github.com/goncalopp/simple-ocr-opencv/blob/master/segmentation_aux.py#L63 Bubbling up the code, we can see that line is making the samples be the detected from the segment's top coordinates. Too few samples thus means too few segments (characters) detected.

You need to figure out how many segments you are getting detected. If you didn't get any, or just a couple, the most likely problem is that your image is defeating the segmentation algorithm.

You can try to run just segmentation by itself by removing pipelines stages here: https://github.com/goncalopp/simple-ocr-opencv/blob/7bcd3f15423321c55584b56eb5635be665784bee/segmentation_filters.py#L8 And running the segmenter like so: https://github.com/goncalopp/simple-ocr-opencv/blob/master/ocr.py#L40

Hopefully you've used pdb before? It's a great tool :) I hope that helps you getting started. Let me know if you need any help (I'll need the image file and your example.py to replicate your results

richard512 commented 8 years ago

Actually, I'm getting errors when attempting to create grounding for digits1.png, too.

What did you use in order to create digits1.box?

Here's the error while attempting to create grounding for digits1.png:

showing after BlurProcessor (waiting for input)
showing ContourSegmenter contours (waiting for input)
showing image after segmentation by RawContourSegmenter (waiting for input)
showing segments filtered by LargeFilter (waiting for input)
showing segments filtered by SmallFilter (waiting for input)
showing segments filtered by LargeAreaFilter (waiting for input)
showing segments filtered by ContainedFilter (waiting for input)
showing line starts and ends (waiting for input)
showing segments filtered by NearLineFilter (waiting for input)
OpenCV Error: Unspecified error (The search tree must be constructed first using train method) in find_nearest, file /build/opencv-SviWsf/opencv-2.4.9.1+dfsg/modules/ml/src/knearest.cpp, line 365
Traceback (most recent call last):
  File "mkground.py", line 14, in <module>
    test_classes, test_segments= ocr.ocr( test_image, show_steps=True )
  File "/home/username/simple-ocr-opencv/ocr.py", line 44, in ocr
    classes= self.classifier.classify( features )
  File "/home/username/simple-ocr-opencv/classification.py", line 61, in classify
    retval, result_classes, neigh_resp, dists= self.knn.find_nearest(features, k= 1)
cv2.error: /build/opencv-SviWsf/opencv-2.4.9.1+dfsg/modules/ml/src/knearest.cpp:365: error: (-2) The search tree must be constructed first using train method in function find_nearest
goncalopp commented 8 years ago

What did you use in order to create digits1.box?

Code very similar to the one on issue 2. I've tried and got it working again.

It looks like you're running ocr.ocr() without calling train() first? If you just want to ground, there's no need to call either.

To be clear, the entire machine learning process needs steps in this order:

If you're having further problems, please publish your code somewhere so I can take a look

richard512 commented 8 years ago

I just found a way to make box files using tesseract:

tesseract digits2.png digits2 batch.nochop makebox

I'm still not sure how to get UserGrounder.ground() inside grounding.py to run because I'm not sure how I'm supposed to supply "segments"

goncalopp commented 8 years ago

I'm still not sure how to get UserGrounder.ground() inside grounding.py to run because I'm not sure how I'm supposed to supply "segments"

A quick read of the source for ocr() should help. I've added example_grounding.py, please let me know if that helps. Instructions on interactive grounding are written to the stdout (command line)

richard512 commented 8 years ago

Loving the new update. This does exactly what I've been searching for. Thank you.

richard512 commented 8 years ago

Forked and added a few things: https://github.com/richard512/simple-ocr-opencv

README update, added a CAPTCHA grounding example, and made a little edit to ContourSegmenter for allowing easier customization of filters.

goncalopp commented 8 years ago

Happy that you found it useful. I did a quick read - looking good. Feel free to do a pull request once you're happy with it, I'll review it more thoroughly Also, if you're breaking CAPTCHAs with this, please let their users know how bad they are :) Hopefully that'll improve the quality.