5-byte padding

[langbeck]

I've a deb package from my local /var/cache/apt/archives. I'm using debian stretch and the package is accountsservice_0.6.43-1_amd64.deb.

If I execute the following commands, everything works (no warnings):

$ ar x accountsservice_0.6.43-1_amd64.deb data.tar.xz
$ xz -d data.tar.xz
$ ls
accountsservice_0.6.43-1_amd64.deb  data.tar
$ tar tf data.tar
./
./etc/
./etc/dbus-1/
./etc/dbus-1/system.d/
./etc/dbus-1/system.d/org.freedesktop.Accounts.conf
...

But if I run the following code (where t/data.tar.xz is the file extracted from the deb cited above), I got an error:

func main() {
    f, err := os.Open("t/data.tar.xz")
    if err != nil {
        panic(err)
    }

    r, err := xz.NewReader(f)
    if err != nil {
        panic(err)
    }

    tr := tar.NewReader(r)
    for {
        hdr, err := tr.Next()
        if err != nil {
            panic(err)
        }   

        log.Println(hdr)
    }
}

The execution output is (I'm using Gogland IDE, but the result is the same if I run from terminal):

GOROOT=/usr/lib/go-1.8
GOPATH=/home/langbeck/git/golang/unpackers/external:/home/langbeck/git/golang/unpackers:/home/langbeck/go
/usr/lib/go-1.8/bin/go build -i -o /tmp/maingo main
/tmp/maingo
panic: xz: unexpected padding size 5

goroutine 1 [running]:
main.main()
    /home/langbeck/git/golang/unpackers/src/main/main.go:127 +0x16c

Note 1: line 127 is the line of the 3rd panic() Note 2: xz --test data.tar.xz && echo OK run fine and the same deb file is installed in my system, so it's a valid deb file.

I'm attaching the deb file in question: accountsservice_0.6.43-1_amd64.deb.zip

[ulikunitz]

Hi, thank you for reporting the problem. The xz specification says that the padding of a block header should be not greater than 3. However it appears that there are files in the wild with a padding of 4 or 5. I have already removed the padding size in the test code and could successfully extract the file.

It will take a little bit longer for a release.

[ulikunitz]

Release 0.5.4 fixes the issue.

[langbeck]

@ulikunitz I use your lib in my work and I must say that it's awesome and I do appreciate the effort that you put in your work. Said that...

I think such restriction doesn't exists. The only restriction seems that all bytes left, after List of Filter Flags had been read, must be zeroes. This seems to be true in all five specs at: https://tukaani.org/xz/format.html

3.1.6. Header Padding

        This field contains as many null byte as it is needed to make
        the Block Header have the size specified in Block Header Size.
        If any of the bytes are not null bytes, the decoder MUST
        indicate an error. It is possible that there is a new field
        present which the decoder is not aware of, and can thus parse
        the Block Header incorrectly.

[ulikunitz]

You are right. It seems that I confused it with the block padding.

3.3. Block Padding

        Block Padding MUST contain 0-3 null bytes to make the size of
        the Block a multiple of four bytes. This can be needed when
        the size of Compressed Data is not a multiple of four. If any
        of the bytes in Block Padding are not null bytes, the decoder
        MUST indicate an error.

I create an issue as a reminder.