JPX stream fails to decode from poppler

GoogleCodeExporter commented 9 years ago

What steps will reproduce the problem?
1. Run our own application that uses Poppler to process a PDF. On a specific 
page of the doc, there is a JPX image that Poppler is unable to decode. While 
OS X's Preview does not render the image when viewing the PDF page, Adobe 
Reader does.
2. Errors reported by Poppler, forwarded from openjpeg:
  Error with COLR Box<0a>
  Failed to decode jp2 structure<0a>
  00000000: expected a marker instead of 0<0a>
  Forbidden value encounter in message header !!<0a>
  [JPT-stream] : Expecting Main header first [class_Id 0] !<0a>
3. Resulting output depends on the version of openjpeg.

What is the expected output? What do you see instead?

Expected: 
- Acrobat Reader shows the image from attached screenshot: 
output-screenshot-acrobat.png
- I can select & copy the image from Reader and then paste it into OS X's 
Preview, but it looks like this fixes whatever issue in the image as, from here 
on, it is rendered correctly by OS X. Exporting from Preview as a JPEG-2000 
generates the attached: output-reader-copy-preview-export.jp2

Actual (within our application, I am extracting whatever stream data I get and 
exporting it as a PNG): 
- OS X, running Poppler linked against openjpeg 1.5.1 (installed via homebrew): 
output-openjpeg-1.5.1-osx-10.8.2.png
- Ubuntu, running Poppler linked against openjpeg 1.3.0 (default apt-get 
version): output-openjpeg-1.3.0-ubuntu-12.04.png
- OS X, running Poppler with built-in JPX decoding (used --disable-libopenjpeg 
option): output-poppler-jpx.png

What version of the product are you using? On what operating system?

Currently forced to stick with 1.5.x. I'm attempting to link against 2.0 but 
this is a ruby extension and I'm fighting some mkmf dependencies that keep 
looking for openjpeg-1. We run this under both OS X 10.8 and Ubuntu linux 12.04.

Please provide any additional information below.

We don't generate these documents; we license them so I'm a bit restricted with 
sharing them. The full document is 106 pages long so I can try and extract that 
single page but I don't have access to Acrobat right now (I should next week). 
We tried to extract the image yesterday but were not successful (I'm not too 
familiar with Acrobat and Preview doesn't handle it correctly when deleting 
other pages). 

I am new to openjpeg and just started looking into this issue in the past 
couple of days.  Tracing down to the spot where the first error happens in 
jp2_read_colr():

    if (cio_tell(cio) - box->init_pos != box->length) 

the value returned by cio_tell(cio) is 108, box->init_pos is 97 but box->length 
is 27 so the comparison fails. I just downloaded 
T-REC-T.808-200501-I!!PDF-E.pdf so I'm trying to become familiar with the 
meaning of everything. For what is worth, at this point in the function, 
jp2->meth is equal to 4 so none of the other cio_tell calls take place but I 
don't know if the trigger of the problem is happening outside of this call.

I'll continue looking at this but figured I should file this in case you'd like 
me to try anything specific since you are more familiar with the inner details. 
Let me know if you need anything else. I'm available on Skype | AIM | gTalk. 
I'm based on EST but most reasonable times should work as I can work from home.

Thank you.

Original issue reported on code.google.com by ed.porras@icloud.com on 8 Mar 2013 at 10:26

Attachments:

GoogleCodeExporter commented 9 years ago

And where is this JPX file?

winfried

Original comment by szukw...@arcor.de on 9 Mar 2013 at 5:03

GoogleCodeExporter commented 9 years ago

I apologize but I wasn't able to get it until today. Here are the two files 
extracted from the page. The first has no issues; the second reports the 
problems I mentioned.

-e

Original comment by ed.porras@icloud.com on 11 Mar 2013 at 5:32

GoogleCodeExporter commented 9 years ago

hi, I just noticed the files are not shown on the web view. I'm attaching both 
here now. The first works; the second has the problem.

Let me know if there's anything you need me to try / do.

Thank you.

Original comment by ed.porras@icloud.com on 12 Mar 2013 at 5:35

Attachments:

GoogleCodeExporter commented 9 years ago

The file p16-2.jp2 has an ICC profile but a meth of 4:

-------------- ICC Profile[0] ---------------
cmmtype(0) version(0.0.0.0) class()
color-space() profile-connection-space(jp2c)
date(44.65361.65359) time(0:0:355)
signature() platform-signature()
profile-flags(0) [0]0 [1]0 [2:15]0
device-attr(0) white(0) negative(0) matte(0) transparent(0)
rendering-intent(0x20701)
CIE: x(263.257) y(65362.12) z(1.1)
-----------------------------------------------
read_colr
    meth(4) prec(2) approx(1) enumcs[0] cs()

The SPEC says:

ISO/IEC 15444-1:2004 (E), Table I.9  Legal METH values

Value       Comment

1           Enumerated Colourspace

2           Restricted ICC profile

other value Reserved for other ISO use. If the value of METH is
            not 1 or 2, there may be fields in this box following the
            APPROX field, and a conforming JP2 reader shall ignore the
            entire Colour Specification box.

I have now made changes for the BRANCH15:

--- openjpeg-branch15-r2301-4/libopenjpeg/jp2.c.v07 2013-03-13 
17:48:08.948600301 +0100
+++ openjpeg-branch15-r2301-4/libopenjpeg/jp2.c 2013-03-13 17:36:45.090567624 
+0100
@@ -665,12 +665,12 @@
   }
    }
    else
-   if(jp2->meth == 3)
+   if(jp2->meth > 2)
    {
 /*  ISO/IEC 15444-1:2004 (E), Table I.9  Legal METH values:
     conforming JP2 reader shall ignore the entire Colour Specification box.
 */
-   cio_skip(cio, box->length);
+   return OPJ_FALSE;
    }

    if (cio_tell(cio) - box->init_pos != box->length)
@@ -797,7 +797,7 @@
 #endif

   /* Part 1, I.5.3.3 : 'must contain at least one' */
-  return (color->jp2_has_colr == 1);
+  return (color->jp2_has_colr == 1 || jp2->meth > 2);

 }/* jp2_read_jp2h() */

'opj_read_colr()' returns OPJ_FALSE so that another COLR
box with a correct METH value can be analyzed.

But now the return value must be changed:

return (color->jp2_has_colr == 1 || jp2->meth > 2);

There has no other COLR box found, and j2k_to_image()
returns with success.

The change for the TRUNK is different:

--- openjpeg-trunk-r2299-5/src/lib/openjp2/jp2.c.v04    2013-03-13 
17:14:05.262502647 +0100
+++ openjpeg-trunk-r2299-5/src/lib/openjp2/jp2.c    2013-03-13 
17:38:54.876573826 +0100
@@ -1135,22 +1135,12 @@

    }
    else
-   if(jp2->meth == 3)
+   if(jp2->meth > 2)
    {
 /* ISO/IEC 15444-1:2004 (E), Table I.9  Legal METH values:
    conforming JP2 reader shall ignore the entire Colour Specification box.
 */
-#ifdef UNUSED_CODE
-   OPJ_INT32 i;
-
-   i = p_colr_header_size - 3;
-
-   while(--i >= 0)
-  {
-   opj_read_bytes(p_colr_header_data,&l_value,1);
-   ++p_colr_header_data;
-  }
-#endif /* UNUSED_CODE */
+   return OPJ_TRUE;
    }
    else
        opj_event_msg(p_manager, EVT_INFO, "COLR BOX meth value is not a regular value (%d), so we will skip the fields following the approx field.\n", jp2->meth);

The UNUSED_CODE can be removed completely, OPJ_TRUE is returned,
and 'opj_decompress()' returns with success because the variable
'color->jp2_has_colr' is only tested in the TRUNK at the beginning
of 'opj_jp2_read_colr()' and then never again.

winfried

Original comment by szukw...@arcor.de on 13 Mar 2013 at 4:41

Attachments:

GoogleCodeExporter commented 9 years ago

I have forgotten to press the STAR button.
winfried

Original comment by szukw...@arcor.de on 13 Mar 2013 at 4:43

GoogleCodeExporter commented 9 years ago

Original comment by mathieu.malaterre on 25 Feb 2014 at 3:30

Added labels: Milestone-Release2.1, Priority-High
Removed labels: Priority-Medium

GoogleCodeExporter commented 9 years ago

This issue was updated by revision r2488.

Original comment by mathieu.malaterre on 26 Feb 2014 at 4:30

GoogleCodeExporter commented 9 years ago

This issue was closed by revision r2489.

Original comment by mathieu.malaterre on 26 Feb 2014 at 4:31

Changed state: Fixed

rossj / openjpeg

JPX stream fails to decode from poppler #211