SetRectangle maybe has an odd thing

[hanoilu]

Today，I want to set a rectangle area to recognition，but I find the parameters may be not explain as baseapi.h . void setrectangle（int left,int top,int width,int height）; when I setrectangle(126,40,1152,28),it will recognizing (0,0,1152,28),I don't know why.

Look forward to your reply，thank you！

[hanoilu]

[bpotard]

Hello,

SetRectangle appears to be broken in v4, cf: https://github.com/sirfz/tesserocr/issues/26

In the meantime, you are probably better off creating a sub-image yourself and performing OCR on it.

[hanoilu]

@bpotard Thank you very much！I'll try~

[bpotard]

A bit more details, with a minor change to the base API to use the SetRectangle API call just after loading an image:

diff --git a/api/baseapi.cpp b/api/baseapi.cpp
index 8b2ef07..a63d9f6 100644
--- a/api/baseapi.cpp
+++ b/api/baseapi.cpp
@@ -1204,6 +1204,7 @@ bool TessBaseAPI::ProcessPage(Pix* pix, int page_index, const char* filename,
   PERF_COUNT_START("ProcessPage")
   SetInputName(filename);
   SetImage(pix);
+  SetRectangle(36, 92, 544, 30);
   bool failed = false;

   if (tesseract_->tessedit_pageseg_mode == PSM_AUTO_ONLY) {

Then run tesseract on testing/phototest.tif.

With branch 3.04:

$ tesseract -psm 6 tesseract/testing/phototest.tif stdout
TIFFFetchNormalTag: Warning, ASCII value for tag "Photoshop" does not end in null byte. Forcing it to be null.
Page 1
This is a lot of 12 point text to test the

TIFFFetchNormalTag: Warning, ASCII value for tag "Photoshop" does not end in null byte. Forcing it to be null.

With master branch:

$tesseract -psm 6 tesseract/testing/phototest.tif stdout
Page 1
s

leptonica-1.74.1 is used in both cases, both on clean debian/jessie64 VMs with identical configurations. ./configure was ran with no option. Without the SetRectangle call, both tesseract versions generate perfect output.

[hanoilu]

@bpotard so,you mean 4.0version have a bug?we can use the portion about "box" of other version instead of 4.0?

[hanoilu]

@bpotard and how free the memory?delete [] utf8text? I run tesseract to handle more than 500 image ,but it tell me mot memory

[bpotard]

yes, you can use version 3.x instead of version 4.0 if you really need to use the SetRectangle call. Alternatively, you can create an image corresponding to the rectangle you want to recognise, and recognise that instead; it won't be exactly equivalent as the bounding boxes would be shifted compared to the ones in the original image, but it is easy to correct the bounding boxes.

[abieler]

it won't be exactly equivalent as the bounding boxes would be shifted compared to the ones in the original image,

@bpotard can you elaborate on why the bounding boxes would be shifted?

[bpotard]

@abieler Because the bounding boxes in each elements (paragraph, word, etc.) would be relative to the "new" sub-image you have created rather than the original image - while setRectangle would normally return bounding boxes relative to the original image. So if you need to know where the recognised text comes from precisely in the original image, you would need to do an additional step to have their exact position: you would need to shift the returned bounding boxes in the original coordinate space... which is admittedly not very hard to do: you just need to add the coordinates of the top left corner of your extracted sub-image to all bounding boxes.

[abieler]

Thanks @bpotard ! I just started using the API and setRectangle and found that the ocr quality is very sensitive to the size of the bounding box, where 1 px more or less on the y axis makes a huge difference, even though there seems no reason by looking at those regions by eye. Is there a "best practice" on how many pixels there should be between the last pixel row of the characters and the bounding box? say, text height is 30 px, then the boundingbox should be 40 px, adding 5 extra pixels on each side (which seems to work ok in my case..) Sorry I should not abuse github issues for this kind of questions...

[bpotard]

If you are using the master branch, SetRectangle is probably still broken so will not work - the bug has not been fixed as far as I know. If you really need the functionality, either use the 3.x branch of tesseract, or create you own sub-images and process them as whole images using the normal API. Do not use SetRectangle in tesseract 4.x.

Alternatively, you can try to figure out where the bug in SetRectangle comes from and fix it :-)

[abieler]

Sorry, that was my mistake, I actually am using sub-images and not setRectangle...

[zdenop]

Can you please provide test case that can demonstrate your problem? I can not reproduce with:

Pix *image = pixRead("/usr/src/testing/phototest.tif");
api->SetImage(image);
api->SetRectangle(36, 126, 582, 31);
outText = api->GetUTF8Text();
printf("Region text:\n%s\n", outText);

and results is:

Region test:
ocr code and see if it works on all types

Which is exactly what e.g. gimp shows for this area. Or do I miss something?

[zdenop]

Closed as not reproducible with current code.

[amitdo]

https://groups.google.com/g/tesseract-ocr/c/PMHq6YSpRRE

@zdenop,

In the linked thread you confirmed that SetRectangle() is not working as expected.

[zdenop]

Notes:

1. Tesseract uses left&bottom coordinate system (0,0) for box files (text2image, tesseract image outputname makebox)
2. SetRectangle was created 14y ago for 3.x and it was always left, top, width, height based. It uses left&top coordinate system (see example below)
3. As far as I tested: it never worked with correctly with LSTM engine, but it works ok with legacy engine.

Here is my test code:

#include <leptonica/allheaders.h>
#include <tesseract/baseapi.h>

int main() {
  // Show version info
  char *versionStrP;
  printf("tesseract %s\n", tesseract::TessBaseAPI::Version());
  versionStrP = getLeptonicaVersion();
  printf(" %s\n", versionStrP);
  lept_free(versionStrP);
  versionStrP = getImagelibVersions();
  printf("  %s\n", versionStrP);
  lept_free(versionStrP);

  tesseract::TessBaseAPI *api = new tesseract::TessBaseAPI();  
  tesseract::OcrEngineMode enginemode = tesseract::OEM_DEFAULT;
  // tesseract::OcrEngineMode enginemode = tesseract::OEM_TESSERACT_ONLY;
  api->Init(NULL, "eng", enginemode);

  Pix *image = pixRead("SetRectangle_test.png");
  api->SetImage(image);
  int w = pixGetWidth(image);
  int h =  pixGetHeight(image);
  int h_adj = h * .3;
  api->SetRectangle(0, 0, w, h_adj);
  char *outTextSR = api->GetUTF8Text();
  printf("********\tOCR output after SetRectangle:\n%s", outTextSR);
  Pix *rect_pix = api->GetThresholdedImage();
  pixWrite("ocred_pix.png", rect_pix, IFF_PNG);

  api->SetImage(rect_pix);
  char *outTextSI = api->GetUTF8Text();
  printf("\n********\tOCR output SetImage:\n%s", outTextSI);

  api->End();
  pixDestroy(&image);
  pixDestroy(&rect_pix);
  delete[] outTextSR;
  delete[] outTextSI;
  delete api;
  return 0;
}

And here is the testing image (SetRectangle_test.png):

With tesseract::OEM_DEFAULT output of GetUTF8Text and GetThresholdedImage will be:

8. testing row
9, testing row

With tesseract::OEM_TESSERACT_ONLY output of GetUTF8Text and GetThresholdedImage will be:

1. testing row
2. testing row

[AdmiralPellaeon]

I also tried to use SetRectangle and got problems with this. Some calls resulted in memory access violation. I noticed that this happend when the image (PNG format) was 8Bit. When storing my image in 32bit depth, then it works. So I suppose that the cannel count isn't properly respected in the method (but only a guess). Would explain the memory access violation when the algorithm operates on a higher dimension that actual is present in memory.

[zdenop]

@AdmiralPellaeon : Is the problem replicable with legacy engine ? There could be 2 different problem for which we should have 2 different issues...

[AdmiralPellaeon]

@zdenop I tested the following:-

• I downloaded trainingdata "eng" from here: https://github.com/tesseract-ocr/tessdata/blob/main/eng.traineddata
• I set tesseract::OEM_TESSERACT_ONLY in the parameter list of init
• result: 32bit png – works fine; 8 Bit image – results in a memory access violation

Did you mean this with the legacy engine (OEM_TESSERACT_ONLY)?

[zdenop]

@AdmiralPellaeon: Yes, I mean OEM_TESSERACT_ONLY. Please create separate issue for it because your problem is related seems to be related to an image format. Please provide a minimal code for reproducing problem + input image that cause problem.

[AdmiralPellaeon]

@zdenop in my opinion it fits in this issue: "SetRectangle maybe has an odd thing".

The image format as source is only a guess of mine. But yes, I can open a new issue if this helps.

[zdenop]

no - "memory access violation" for 8bit image does not fit to issue....