search

gotomypc / leptonica

Automatically exported from code.google.com/p/leptonica
0 stars 0 forks source link

Include regression tests in README #81

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Download leptonica-1.69.tar.gz
2. un-tar and look at README.
3. run autoconf, configure, make
4. Empirically, this builds the programs in prog.  The README says
   the programs in the prog directory are regression tests and
   example programs, but I don't find anything that says how to
   run the regression tests.

What is the expected output? What do you see instead?
I would expect that the README would say something like:
To compile: make
To install: make install
To run regression tests:  make regress  (or make test or ...)
but I didn't find anything 

What version of the product are you using? On what operating system?
1.69 on Fedora Core 17 Linux on an AMD 64-bit processor.

Please provide any additional information below.

I found "alltests_reg"  (not sure if "reg" is "register" or 'regression")
and ran it.  It seemed to be the way to run the regressions.  The
"alltests_reg compare" command eventually says 
Success in 37 of 46 *_reg programs (output matches the "golden" files)
Time for all regression tests: 74 sec
which suggests something is not running right, and I get things
like:

80 Warnings
      1 Warning in getScaledParametersPS: right edge > 8.5 inch
     12 Warning in getSortedPathnamesInDirectory: no files found
     53 Warning in pixConvertTo8Or32: pix has colormap; removing
      1 Warning in pixModifySaturation: no change requested in saturation
      2 Warning in pixSauvolaBinarizeTiled: tile height too small; ny reduced to 28
        ...
92 Errors
     40 Error in fopenReadStream: file not found
     40 Error in nbytesInFile: stream not opened
      2 Error in regTestCompareFiles: golden file projection_golden.1. not found
      2 Error in regTestCompareFiles: golden file projection_golden.10. not found
        ...

124 Failures
      2 Failure in rankhisto_reg, index 4: comparing /tmp/rankhisto.4.png with /tmp/golden/rankhisto_golden.4.png
      2 Failure in rankhisto_reg, index 2: comparing /tmp/rankhisto.2.png with /tmp/golden/rankhisto_golden.2.png
      2 Failure in rankbin_reg, index 3: comparing /tmp/h_30bin.png with /tmp/golden/rankbin_golden.3.png
      2 Failure in rankbin_reg, index 2: comparing /tmp/w_30bin.png with /tmp/golden/rankbin_golden.2.png
        ...

But I don't know what to do about these. Are they coding errors or environmental
errors or configuration errors or ...?

It would help if there were some notes, suggestions, explanations in the README.

Original issue reported on code.google.com by l...@austin.rr.com on 22 Jul 2013 at 8:51

GoogleCodeExporter commented 9 years ago 
Thank you for your report.  I agree that there should be an explanation in the 
README of how to use the regression tests.  This will be added for 1.70.

The regression tests covered by alltests_reg include about 40% of the programs 
in the prog directory.

You need to first run
     alltests_reg generate
to generate the golden files in /tmp/golden/
(suggest you make this directory first, just in case ...)

Then you can run 
    alltests_reg compare    (to compare results with golden files)

You can also run
    alltests_reg display   (to throw a lot of results up on your display)

The warnings you saw are not a problem.  The errors were because there were no 
golden files.  I'm surprised you got 37/46 correct.  That may be a bug; if so, 
it will be fixed in 1.70.

Original comment by dan.bloo...@gmail.com on 23 Jul 2013 at 4:07

GoogleCodeExporter commented 9 years ago 
Excellent.  Running "alltests_reg generate" produces 68 warnings:

     53 Warning in pixConvertTo8Or32: pix has colormap; removing
      5 Warning in pixWriteStreamBmp: writing 2 bpp bmp file; nobody else can read
      2 Warning in pixSauvolaBinarizeTiled: tile height too small; ny reduced to 78
      2 Warning in pixSauvolaBinarizeTiled: tile height too small; ny reduced to 28
      1 Warning: empty y range [99:99], adjusting to [98.01:99.99]

After this, running "alltests_reg compare" generates
68 Warnings, 15 Failures and 0 Errors:

      1 FAILURE: findpattern_reg
      1 Failed to complete findpattern_reg
      2 Failure in findpattern_reg, index 11: comparing /tmp/findpattern.11.png with /tmp/golden/findpattern_golden.11.png
      2 Failure in findpattern_reg, index 15: comparing /tmp/findpattern.15.png with /tmp/golden/findpattern_golden.15.png
      2 Failure in findpattern_reg, index 17: comparing /tmp/findpattern.17.png with /tmp/golden/findpattern_golden.17.png
      2 Failure in findpattern_reg, index 19: comparing /tmp/findpattern.19.png with /tmp/golden/findpattern_golden.19.png
      2 Failure in findpattern_reg, index 3: comparing /tmp/findpattern.3.png with /tmp/golden/findpattern_golden.3.png
      2 Failure in findpattern_reg, index 5: comparing /tmp/findpattern.5.png with /tmp/golden/findpattern_golden.5.png
      2 Failure in findpattern_reg, index 7: comparing /tmp/findpattern.7.png with /tmp/golden/findpattern_golden.7.png

     53 Warning in pixConvertTo8Or32: pix has colormap; removing
      5 Warning in pixWriteStreamBmp: writing 2 bpp bmp file; nobody else can read
      2 Warning in pixSauvolaBinarizeTiled: tile height too small; ny reduced to 78
      2 Warning in pixSauvolaBinarizeTiled: tile height too small; ny reduced to 28
      1 Warning: empty y range [99:99], adjusting to [98.01:99.99]
      1 Warning: empty y range [51:51], adjusting to [50.49:51.51]
      1 Warning: empty y range [1:1], adjusting to [0.99:1.01]
      1 Warning: empty y range [0:0], adjusting to [-1:1]
      1 Warning in pixModifySaturation: no change requested in saturation
      1 Warning in getScaledParametersPS: right edge > 8.5 inch

So I'm guessing only the Failures matter, which are for findpattern_reg
Success in 45 of 46 *_reg programs (output matches the "golden" files)

Original comment by l...@austin.rr.com on 23 Jul 2013 at 2:31

GoogleCodeExporter commented 9 years ago 
Lyle, thanks for checking this out.

findpattern_reg runs in 1.70 (beta) without errors, but we'll double check on 
the distro.

BTW, can you get findpattern_reg to run if you don't give it any arguments?  It 
is a nice demonstration of using hit-miss transforms in a pattern-recognition 
application.

Original comment by dan.bloo...@gmail.com on 23 Jul 2013 at 8:40

GoogleCodeExporter commented 9 years ago 
I figured the purpose of the regressions was to check if my
configuration and build worked correctly. Detect chip and
compiler issues or library problems, for example.

I can run "findpattern_reg" with no arguments, and I get:

> findpattern_reg

################   findpattern_reg   ###############
leptonica-1.69
libgif 4.1.6 : libjpeg 6b : libpng 1.5.10 : libtiff 3.9.7 : zlib 1.2.5

Warning in pixConvertTo8Or32: pix has colormap; removing
sh: xzgv: command not found
Time to find patterns =   0.013
Warning in pixConvertTo8Or32: pix has colormap; removing
sh: xzgv: command not found
Warning in pixConvertTo8Or32: pix has colormap; removing
sh: xzgv: command not found
Time to find patterns =   0.006
sh: xzgv: command not found
Warning in pixConvertTo8Or32: pix has colormap; removing
sh: xzgv: command not found
Time to find patterns =   0.001
sh: xzgv: command not found
Warning in pixConvertTo8Or32: pix has colormap; removing
sh: xzgv: command not found
Time to find patterns =   0.004
Warning in pixConvertTo8Or32: pix has colormap; removing
sh: xzgv: command not found
Warning in pixConvertTo8Or32: pix has colormap; removing
sh: xzgv: command not found
Time to find patterns =   0.001
sh: xzgv: command not found
Time:   1.084 sec
################################################

If I run "yum provides xzgv", it should be in package xzgv-0.9.1-5.fc17.x86_64,
which I can install, to get rid of the "command not found" messages, and
creates a lot of windows of the Rome News Tribune.

If I try to compare the output /tmp/findpattern.7.png with the golden
/tmp/golden/findpattern_golden.7.png, they appear to be the same image
(I can use "convert" to convert both of them to GIFs and these are
identical files), but cmp says that they differ. Looking up the PNG
file format, and writing a small program to dump the structure, it
appears the first two chunks are the same (IHDR, pHYs), but they
differ in the 3rd chunk (IDAT) and then the following ones too.
At this point, I'm over my head.

Original comment by l...@austin.rr.com on 24 Jul 2013 at 12:33

GoogleCodeExporter commented 9 years ago 
Lyle, 1.70 is released.  You can get it at:  leptonica.org/download.html

I'm closing this bug.  There are occasional differences in the file bytes, even 
without any difference in the image.  I will continue to make these tests more 
robust.  The new release lets you use the automake to build and test (see 
Makefile.html).

Original comment by dan.bloo...@gmail.com on 23 Jan 2014 at 8:54