search

libnxz / power-gzip

POWER NX zlib compliant library
23 stars 18 forks source link

EOFException when java nxgzip testing #203

Open chenhealth1234 opened 1 year ago

chenhealth1234 commented 1 year ago

OS：Centos 8.5 kernel：4.18.0-348.el8.ppc64le jdk version：jdk-11.0.19-7

when java nxgzip-v0.64 testing, we met EOF issue, the issue is as follow:

LD_PRELOAD=./libnxz.so.0.0.64 java ShortRead size=14 csize=19 available=14 Exception in thread "main" java.io.EOFException: Unexpected end of ZLIB input stream at java.base/ja4va.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:467) at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:159) at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107) at ShortRead.main(ShortRead.java:64)

As this issue occur, we did libz and nxgzip-version0.63 testing, found this issue not exist, so we check the data and found the data by nxgzip0.64 is not right, the detail is as follow:

Libz testing:

cd jdk11u-jdk-11.0.19-7/test/jdk/java/util/zip/ZipFile/

javac ShortRead.java

java ShortRead

size=14 csize=16 available=14 uncompressdata:Data disponibl

xxd abc.zip

00000000: 504b 0304 1400 0808 0800 0e75 dd56 0000 PK.........u.V.. 00000010: 0000 0000 0000 0000 0000 0300 0000 6162 ..............ab 00000020: 6373 492c 4954 48c9 2c2e c8cf cb4c ca01 csI,ITH.,....L.. 00000030: 0050 4b07 084f ea5b 2f10 0000 000e 0000 .PK..O.[/....... 00000040: 0050 4b01 0214 0014 0008 0808 000e 75dd .PK...........u. 00000050: 564f ea5b 2f10 0000 000e 0000 0003 0000 VO.[/........... 00000060: 0000 0000 0000 0000 0000 0000 0000 0061 ...............a 00000070: 6263 504b 0506 0000 0000 0100 0100 3100 bcPK..........1. 00000080: 0000 4100 0000 0000 ..A.....

nxgzip 0.63 testing:

LD_PRELOAD=./libnxz.so.0.0.63 java ShortRead

size=14 csize=16 available=14 uncompressdata:Data disponibl

xxd abc.zip

00000000: 504b 0304 1400 0808 0800 0e75 dd56 0000 PK.........u.V.. 00000010: 0000 0000 0000 0000 0000 0300 0000 6162 ..............ab 00000020: 6373 492c 4954 48c9 2c2e c8cf cb4c ca01 csI,ITH.,....L.. 00000030: 0050 4b07 084f ea5b 2f10 0000 000e 0000 .PK..O.[/....... 00000040: 0050 4b01 0214 0014 0008 0808 000e 75dd .PK...........u. 00000050: 564f ea5b 2f10 0000 000e 0000 0003 0000 VO.[/........... 00000060: 0000 0000 0000 0000 0000 0000 0000 0061 ...............a 00000070: 6263 504b 0506 0000 0000 0100 0100 3100 bcPK..........1. 00000080: 0000 4100 0000 0000 ..A.....

nxgzip 0.64 testing:

LD_PRELOAD=./libnxz.so.0.0.64 java ShortRead

size=14 csize=19 available=14 Exception in thread "main" java.io.EOFException: Unexpected end of ZLIB input stream at java.base/ja4va.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:467) at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:159) at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107) at ShortRead.main(ShortRead.java:64)

xxd abc.zip

00000000: 504b 0304 1400 0808 0800 2f74 dd56 0000 PK......../t.V.. 00000010: 0000 0000 0000 0000 0000 0300 0000 6162 ..............ab 00000020: 6301 0e00 f1ff 4461 7461 2064 6973 706f c.....Data dispo 00000030: 6e69 626c 504b 0708 4fea 5b2f 1300 0000 niblPK..O.[/.... 00000040: 0e00 0000 504b 0102 1400 1400 0808 0800 ....PK.......... 00000050: 2f74 dd56 4fea 5b2f 1300 0000 0e00 0000 /t.VO.[/........ 00000060: 0300 0000 0000 0000 0000 0000 0000 0000 ................ 00000070: 0000 6162 6350 4b05 0600 0000 0001 0001 ..abcPK......... 00000080: 0031 0000 0044 0000 0000 00 .1...D.....

javac ShortRead.java

the testing source code: ShortRead.java

/*

/**

import java.io.; import java.util.zip.; import java.io.IOException;

public class ShortRead {

public static void main(String[] args) throws Exception {
    final File zFile = new File("abc.zip");
    try {
        final String entryName = "abc";
        final String data = "Data disponibl";

/ Process proc = null; // zip压缩文件 proc = Runtime.getRuntime().exec("zip -r abc.zip abc"); proc.waitFor(); / try (FileOutputStream fos = new FileOutputStream(zFile); ZipOutputStream zos = new ZipOutputStream(fos)) { zos.putNextEntry(new ZipEntry(entryName)); zos.write(data.getBytes("ASCII")); zos.closeEntry(); }

        try (ZipFile zipFile = new ZipFile(zFile)) {
            final ZipEntry zentry = zipFile.getEntry(entryName);
            final InputStream inputStream = zipFile.getInputStream(zentry);
            System.out.printf("size=%d csize=%d available=%d%n",
                              zentry.getSize(),
                              zentry.getCompressedSize(),
                              inputStream.available());
            byte[] buf = new byte[data.length()];
            final int count = inputStream.read(buf);

            System.out.printf("uncompressdata:%s\n",new String(buf, "ASCII"));

            if (! new String(buf, "ASCII").equals(data) ||
                count != data.length())
                throw new Exception("short read?");
        }
    } finally {

// zFile.delete(); } } }

RajalakshmiSR commented 1 year ago

Looks like the difference in behavior is introduced in commit https://github.com/libnxz/power-gzip/commit/94573fedddd46a24687f0d6489f434cb1333b3f7#

soo207 commented 1 year ago

Test InflaterBufferSize.java

[root@113 zip]# pwd
/root/jdk11u-jdk-11.0.19-7/test/jdk/java/util/zip
[root@113 zip]# cat testfile
1234567890

1. Libz testing

[root@113 zip]# javac InflaterBufferSize.java
[root@113 zip]# java InflaterBufferSize -1 testfile
******************************
Original data length: 11 bytes
level = -1
Deflated data length: 19 bytes
Inflated data length: 11 bytes

Passed = 2 failed = 0

2. nxgzip 0.64 testing

[root@113 zip]# LD_PRELOAD=/libnxz.so.0.0.64 java InflaterBufferSize -1 testfile
******************************
Original data length: 11 bytes
level = -1
Deflated data length: 19 bytes
Inflated data length: 0 bytes
Inflated and deflated arrays do not match
java.lang.Exception: Stack trace
        at java.base/java.lang.Thread.dumpStack(Thread.java:1383)
        at InflaterBufferSize.fail(InflaterBufferSize.java:155)
        at InflaterBufferSize.fail(InflaterBufferSize.java:156)
        at InflaterBufferSize.check(InflaterBufferSize.java:161)
        at InflaterBufferSize.realMain(InflaterBufferSize.java:140)
        at InflaterBufferSize.main(InflaterBufferSize.java:166)

Passed = 1 failed = 1
Exception in thread "main" java.lang.AssertionError: Some tests failed
        at InflaterBufferSize.main(InflaterBufferSize.java:168)

3. nxgzip 0.64 testing(nx_selector = 3,Use nx to compress and zlib to decompress)

[root@113 zip]# LD_PRELOAD=/libnxz.so.0.0.64 java InflaterBufferSize -1 testfile
******************************
Original data length: 11 bytes
level = -1
Deflated data length: 22 bytes
Inflated data length: 11 bytes

Passed = 2 failed = 0
RajalakshmiSR commented 1 year ago

Can you add the test case?

charleschen2000 commented 1 year ago

FYI. All the related test cases can be found at: https://github.com/openjdk/jdk11u/tree/jdk-11.0.19%2B7/test/jdk/java/util/zip

charleschen2000 commented 1 year ago

FYI. All the related test cases can be found at: https://github.com/openjdk/jdk11u/tree/jdk-11.0.19%2B7/test/jdk/java/util/zip

InflaterBufferSize.java: https://github.com/openjdk/jdk11u/blob/jdk-11.0.19%2B7/test/jdk/java/util/zip/InflaterBufferSize.java

RajalakshmiSR commented 1 year ago

Is this the first time these test cases are executed with libnxz?

charleschen2000 commented 1 year ago

YES, it is the first time we run those tests with libnxz.
BTW, those test cases are intended for java.util.zip regression test, and we can invoke all of them using jtreg as mentioned in #205 .

sachinmonga64 commented 1 year ago

Hi

Request you to please close the tickets https://github.com/libnxz/power-gzip/issues/203, https://github.com/libnxz/power-gzip/issues/204 and https://github.com/libnxz/power-gzip/issues/205 if the issues are resolved with the latest shared patches.

Regards: Sachin.