michaelrsweet commented 17 years ago

Version: 1.3.0 CUPS.org User: kssingvo.suse

Download this PPD file: http://www.linuxprinting.org/download/PPD/Infotec/PS/Infotec-ISC_2432_PS.ppd

compress it: gzip -9 Infotec-ISC_2432_PS.ppd

run cupstestppd:

cupstestpppd Infotec-ISC_2432_PS.ppd.gz Infotec-ISC_2432_PS.ppd.gz: FAIL FAIL Unable to open PPD file - Missing value string on line 1869.

REF: Page 20, section 3.4.

Well, the issue is not that the file cannot be opened: a specific line number is given, so program is able to open it.

I got the feeling that this depends on the size before/after compression and that not enough memory gets allocated for such a compressed file. But I'm not sure.

michaelrsweet commented 17 years ago

CUPS.org User: kssingvo.suse

Before I forget to mention:

using file uncompressed, cupstestppd doesn't fail (only a warning)
using files with similar size, produces same error
using files with larger/smaller size doesn't produce that error

michaelrsweet commented 17 years ago

CUPS.org User: kssingvo.suse

I noticed that this is 1.2.12 related, and in 1.3.0 fixed. Sorry.

michaelrsweet commented 17 years ago

CUPS.org User: kssingvo.suse

Umpf. Sorry. I noticed now, that I accidently overwrote my testfile. :( Issue is still present in 1.3.0. :(

michaelrsweet commented 17 years ago

CUPS.org User: mike

I can't reproduce this on my machine with current 1.3.x or 1.4.x.

Does the "testfile" unit test program show any problems (located in the "cups" subdirectory) after you build CUPS?

michaelrsweet commented 17 years ago

CUPS.org User: kssingvo.suse

No, "testfile" doesn't report anything special.

But (!) the output of "testfile" is incomplete. It ends in the reported line 1869, and everything past "*FoomaticRIPOptionSet" is missing (= around 10 lines).

michaelrsweet commented 17 years ago

CUPS.org User: mike

What version of zlib are you using on that system?

michaelrsweet commented 17 years ago

CUPS.org User: pipitas

WORKSFORME.

hjssuse:~ > rpm -qif $(which cupstestppd) Name : cups Relocations: (not relocatable) Version : 1.2.11 Vendor: openSUSE Build Service Release : 7.1 Build Date: Thu 10 May 2007 02:50:39 PM CEST Install Date: Thu 05 Jul 2007 03:08:06 PM CEST Build Host: build09 Group : Hardware/Printing Source RPM: cups-1.2.11-7.1.src.rpm Size : 10798222 License: GNU General Public License (GPL) Signature : DSA/SHA1, Thu 10 May 2007 02:53:45 PM CEST, Key ID 3b3011b76b9d6523

hjssuse:~ > rpm -q zlib zlib-1.2.3-33

Kurt Pfeifle System & Network Printing Consultant ---- Linux/Unix/Windows/Samba/CUPS Infotec Deutschland GmbH ..................... Hedelfinger Strasse 58 A RICOH Company ........................... D-70327 Stuttgart/Germany

michaelrsweet commented 17 years ago

CUPS.org User: mike

Also, if you run "testfile" without any arguments it will do some read/write tests to eliminate possible data loss/corruption issues.

I'd still like to know what version of zlib you have, along with the arch in case this is specific to a version of zlib or processor...

michaelrsweet commented 17 years ago

CUPS.org User: kssingvo.suse

Running zlib-1.2.3 Last changes were done in July 2005.

michaelrsweet commented 17 years ago

CUPS.org User: kssingvo.suse

A few additional thoughts:

this issue happens on 64bit as well as on 32bit
cupstestppd and testppd stop output at the same position
we noticed this issue due to a change of our check script: previously we used cat/zcat and a pipe for cupstestppd, now we're removed the pipe mechanism and read file directly from program. Therefore this can be an issue since a long time, and nobody noticed it yet
it happens only on few files, and their size is near buffer size of 64kb, which can point toward an issue in calloc or similar
hadn't have much time to dig deeper into it today, too much interrupts from other work. And tommorrow I'm only for short in office

michaelrsweet commented 17 years ago

CUPS.org User: mike

What do you get if you run "testfile" by itself?

I haven't have any reports of problems since early in the 1.2 betas, and it sounds like Kurt is running the same SuSE release and also doesn't have the same problem. I would also expect problems with the help index (/var/cache/cups/help.index) if there is a problem with our interface with zlib.

What version of gzip are you using?

michaelrsweet commented 17 years ago

CUPS.org User: kssingvo.suse

I'm not quiet sure, what "by itself" exactly means. But maybe this:

./testfile cupsFileOpen(write): FAIL (Permission denied) cupsFileOpen(read): FAIL (No such file or directory)

cupsFileOpen(write compressed): FAIL (Permission denied) cupsFileOpen(read): FAIL (No such file or directory) cupsFileFind: PASS (/bin/cat)

4 TEST(S) FAILED!

(please specify more details if I got it wrong)

The help index isn't mentioned in /var/log/cups/error_log*

I can run "cupstestppd" in a chroot of SUSE 10.0 with cups-1.1.23.

and I get a PASS

cupstestppd Infotec-ISC_2432_PS.ppd.gz

Infotec-ISC_2432_PS.ppd.gz: PASS

I can run "cupstestppd" in a chroot of openSUSE 10.2 with cups-1.2.7

and I get a FAIL

cupstestppd Infotec-ISC_2432_PS.ppd.gz Infotec-ISC_2432_PS.ppd.gz: FAIL FAIL Unable to open PPD file - Missing value string on line 1869.

REF: Page 20, section 3.4.

gzip version is: 1.3.12 in current system gzip version is: 1.3.5 in openSUSE 10.2 gzip version is: 1.3.5 in SUSE 10.0

zlib is in every system: 1.2.3

michaelrsweet commented 17 years ago

CUPS.org User: kssingvo.suse

Ah... and Kurt seems to be running a late alpha version of upcoming openSUSE 10.3. I'm running beta2 of openSUSE 10.3

(Note: 1.3.0 came after feature freeze for openSUSE 10.3. So we will ship 1.2.12 in this product. Nevertheless I'm able to test with 1.3.0 :-)

michaelrsweet commented 17 years ago

CUPS.org User: mike

Yes, run "testfile" with no arguments. However, you need write access to that directory to make the tests worthwhile/useful.

I'm testing on Mac OS X 10.5 and Fedora 7, both with gzip 1.3.x and zlib 1.2.3 and both on 32-bit Intel processors.

Have you tested this on other machines, just in case there is a problem with yours?

michaelrsweet commented 17 years ago

CUPS.org User: mike

Also, can you post your gzip'd copy of the PPD file?

michaelrsweet commented 17 years ago

CUPS.org User: mike

Hmm, even that file works for me...

Any chance you are using a new version of GCC, or some heavy optimization flags?

michaelrsweet commented 17 years ago

CUPS.org User: kssingvo.suse

After I did several tests, like trying to run cupstestppd within a clean openSUSE 10.2 system (booted my machine accordinly), etc. I didn't find anything.

But now I found the cause of the trouble: it's the file itself. There is no final linebreak (vim says: [noeol]) at the end of the file. If I add this final linebreak, and compress it again, it passes the test as expected.

So, how can we improve cupstestppd here? Or is it a zlib issue? Do you need additional information, like debug output from cups library?

michaelrsweet commented 17 years ago

CUPS.org User: mike

This will probably end up being an issue in the cupsFile APIs; I'll add another test to testfile.c to see if I can duplicate the problem with a final line without CR or LF.

michaelrsweet commented 17 years ago

CUPS.org User: kssingvo.suse

If this helps: the related code where the fp->eof in cups/file.c is marked as true is this: if (tcrc != fp->crc) { / * Bad CRC, mark end-of-file... / fp->eof = 1;

        return (-1);
      }

Adding some debug output, I see that tcrc=-1600534055, and crc=0 (32bit machine)

michaelrsweet commented 17 years ago

CUPS.org User: mike

Hmm, adding a partial line test in testfile.c doesn't reproduce the problem, but I am finally able to reproduce it with your test file...

michaelrsweet commented 17 years ago

CUPS.org User: mike

OK, turns out we were not computing the CRC as we read the data, leading to this error. Fix coming soon...

michaelrsweet commented 17 years ago

CUPS.org User: mike

Fixed in Subversion repository.

michaelrsweet commented 17 years ago

"str2510.patch":

Index: testfile.c

--- testfile.c (revision 6961) +++ testfile.c (working copy) @@ -16,7 +16,8 @@ *

Contents: *
- * main() - Main entry.
- * main() - Main entry.
- * read_write_tests() - Perform read/write tests. */
/ @@ -149,6 +150,8 @@ unsigned char readbuf[8192], /* Read buffer / writebuf[8192]; / Write buffer / int byte; / Byte from file /
- static const char *partial_line = "partial line";
- /* Partial line */
/* @@ -253,11 +256,11 @@

fputs("cupsFileWrite(): ", stdout);
for (i = 0; i < 100; i ++)
for (i = 0; i < 10000; i ++) if (cupsFileWrite(fp, (char *)writebuf, sizeof(writebuf)) < 0) break;
if (i >= 100)
if (i >= 10000) puts("PASS"); else { @@ -266,6 +269,20 @@ }

/*
* cupsFilePuts() with partial line...
*/ +
fputs("cupsFilePuts(\"partial line\"): ", stdout); +
if (cupsFilePuts(fp, partial_line) > 0)
puts("PASS");
else
{
printf("FAIL (%s)\n", strerror(errno));
status ++;
} +
/*
- cupsFileClose() */

@@ -405,13 +422,13 @@

 fputs("cupsFileRead(): ", stdout);

for (i = 0; i < 100; i ++)
for (i = 0; i < 10000; i ++) if ((byte = cupsFileRead(fp, (char *)readbuf, sizeof(readbuf))) < 0) break; else if (memcmp(readbuf, writebuf, sizeof(readbuf))) break;
if (i >= 100)
if (i >= 10000) puts("PASS"); else if (byte > 0) { @@ -431,6 +448,26 @@ }

/*
* cupsFileGetChar() with partial line...
*/ +
fputs("cupsFileGetChar(partial line): ", stdout); +
for (i = 0; i < strlen(partial_line); i ++)
if ((byte = cupsFileGetChar(fp)) < 0)
break;
else if (byte != partial_line[i])
break; +
if (!partial_line[i])
puts("PASS");
else
{
printf("FAIL (got '%c', expected '%c')\n", byte, partial_line[i]);
status ++;
} +
/*
- cupsFileClose() */

Index: file.c

--- file.c (revision 6961) +++ file.c (working copy) @@ -1207,7 +1207,7 @@ if (fp->ptr >= fp->end) if (cups_fill(fp) <= 0) {

DEBUG_printf((" cups_fill() returned -1, total=%d\n", total));

 DEBUG_printf(("    cups_fill() returned -1, total=%d\n", (int)total));

if (total > 0)
 return ((ssize_t)total);

@@ -1235,7 +1235,7 @@

Return the total number of bytes read... */

DEBUG_printf((" total=%d\n", total));
DEBUG_printf((" total=%d\n", (int)total));

return ((ssize_t)total); } @@ -1703,6 +1703,7 @@ { ssize_t bytes; /* Number of bytes read */

ifdef HAVE_LIBZ
int status; /* Decompression status / const unsigned char *ptr, / Pointer into buffer / *end; / End of buffer _/
endif /_ HAVE_LIBZ */

@@ -1921,7 +1922,13 @@ fp->stream.next_out = (Bytef *)fp->buf; fp->stream.avail_out = sizeof(fp->buf);

if (inflate(&(fp->stream), Z_NO_FLUSH) == Z_STREAM_END)

status = inflate(&(fp->stream), Z_NO_FLUSH); +
if (fp->stream.next_out > (Bytef *)fp->buf)
fp->crc = crc32(fp->crc, (Bytef *)fp->buf,
fp->stream.next_out - (Bytef *)fp->buf); +
```
if (status == Z_STREAM_END)
```
{ /*
- Read the CRC and length... @@ -1950,6 +1957,9 @@ * Bad CRC, mark end-of-file... */

     DEBUG_printf(("cups_fill: tcrc=%08x, fp->crc=%08x\n",

              (unsigned int)tcrc, (unsigned int)fp->crc));

+ fp->eof = 1;

return (-1);

apple / cups

cupstestppd fails with misleading message #2510

run cupstestppd:

REF: Page 20, section 3.4.

I'm not quiet sure, what "by itself" exactly means. But maybe this:

4 TEST(S) FAILED!

and I get a PASS

Infotec-ISC_2432_PS.ppd.gz: PASS

and I get a FAIL

REF: Page 20, section 3.4.

Index: testfile.c

Index: file.c

ifdef HAVE_LIBZ

endif /_ HAVE_LIBZ */