search

jmacd / xdelta

open-source binary diff, delta/differential compression tools, VCDIFF/RFC 3284 delta compression
http://xdelta.org
1.09k stars 181 forks source link

Can't switch off secondary compression #234

Open zonque opened 7 years ago

zonque commented 7 years ago

I can't seem to switch off secondary compression for generated image files. xdelta3 printhdr keep claiming the image has secondary compression even when I force it off with -S and no additional argument.

There's also a similar issue with the python wrapper which might be related. The details are here: https://github.com/samuelcolvin/xdelta3-python/issues/1

mgrinzPlayer commented 7 years ago 
-S none
zonque commented 7 years ago

Doesn't help:

$ xdelta3 -S none -s a b delta

$ xdelta3 printhdr delta
VCDIFF version:               0
VCDIFF header size:           21
VCDIFF header indicator:      VCD_APPHEADER 
VCDIFF secondary compressor:  lzma
VCDIFF application header:    rootfs//rootfs/
XDELTA filename (output):     rootfs
XDELTA filename (source):     rootfs
VCDIFF window number:         0
VCDIFF window indicator:      VCD_SOURCE VCD_ADLER32 
VCDIFF adler32 checksum:      A0CFD3B3
VCDIFF copy window length:    8660193
VCDIFF copy window offset:    0
VCDIFF delta encoding length: 128517
VCDIFF target window length:  8388608
VCDIFF data section length:   127735
VCDIFF inst section length:   472
VCDIFF addr section length:   294
$ xdelta3 -V
Xdelta version 3.1.0, Copyright (C) Joshua MacDonald
Xdelta comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; see "COPYING" for details.
zonque commented 7 years ago

Does this work for you?

mgrinzPlayer commented 7 years ago

I'm 100% sure that "-S none" worked in 3.0.8. You can try 3.0.8 xdelta3 version? (you can download it from https://github.com/jmacd/xdelta/tree/ee616b958d794c6714d117b22f0be6757b379d7e/xdelta3/releases)

samuelcolvin commented 7 years ago

I've just tried on release3_1_apl which xdelta3-python uses.

As far as I understand the above works fine:

(env) xdelta3 ➤  ./xdelta3 printhdr delta
VCDIFF version:               0
VCDIFF header size:           11
VCDIFF header indicator:      VCD_APPHEADER 
VCDIFF secondary compressor:  none
VCDIFF application header:    b//a/
XDELTA filename (output):     b
XDELTA filename (source):     a
VCDIFF window number:         0
VCDIFF window indicator:      VCD_SOURCE VCD_ADLER32 
VCDIFF adler32 checksum:      C5011D33
VCDIFF copy window length:    71
VCDIFF copy window offset:    9
VCDIFF delta encoding length: 22
VCDIFF target window length:  80
VCDIFF data section length:   9
VCDIFF inst section length:   3
VCDIFF addr section length:   1
(env) xdelta3 ➤  ./xdelta3 -V
Xdelta version 3.1.1, Copyright (C) Joshua MacDonald
Xdelta comes with ABSOLUTELY NO WARRANTY.
Licensed under the Apache License, Version 2.0
See "LICENSE" for details.
(env) xdelta3 ➤  git status
On branch release3_1_apl
Your branch is up-to-date with 'origin/release3_1_apl'.
...
zonque commented 7 years ago

@samuelcolvin are you compiling this with lzma support?

samuelcolvin commented 7 years ago

I think not.

zonque commented 7 years ago

@mgrinzPlayer good idea. I checked 3.0.9 and it worked, then I did a quick bisect and it turns out e1a0a2538f is the first bad commit.

That commits takes lzma as default compressor if it is compiled in, but breaks explicit -S none runtime configurations.

That of course means that users that don't compile with lzma support aren't affected.

/cc the author of that commit, @jmacd

mgrinzPlayer commented 7 years ago 
D:\>xdelta3-3.0.11-i686.exe printhdr lol.patch
VCDIFF version:               0
VCDIFF header size:           105
VCDIFF header indicator:      VCD_APPHEADER
VCDIFF secondary compressor:  lzma
VCDIFF application header:    gry\Prey\GameSDK\Precache\patch.pak//SteamLibrary\steamapps\common\Prey\GameSDK\Precache\patch.pak/
XDELTA filename (output):     gry\Prey\GameSDK\Precache\patch.pak
XDELTA filename (source):     SteamLibrary\steamapps\common\Prey\GameSDK\Precache\patch.pak
VCDIFF window number:         0
VCDIFF window indicator:      VCD_ADLER32
VCDIFF adler32 checksum:      CB266ECB
VCDIFF delta encoding length: 16777236
VCDIFF target window length:  16777216
VCDIFF data section length:   16777216
VCDIFF inst section length:   5
VCDIFF addr section length:   0

D:\>xdelta3-3.0.8.x86-32.exe printhdr lol.patch
VCDIFF version:               0
VCDIFF header size:           105
VCDIFF header indicator:      VCD_APPHEADER
VCDIFF secondary compressor:  none
VCDIFF application header:    gry\Prey\GameSDK\Precache\patch.pak//SteamLibrary\steamapps\common\Prey\GameSDK\Precache\patch.pak/
XDELTA filename (output):     gry\Prey\GameSDK\Precache\patch.pak
XDELTA filename (source):     SteamLibrary\steamapps\common\Prey\GameSDK\Precache\patch.pak
VCDIFF window number:         0
VCDIFF window indicator:      VCD_ADLER32
VCDIFF adler32 checksum:      CB266ECB
VCDIFF delta encoding length: 16777236
VCDIFF target window length:  16777216
VCDIFF data section length:   16777216
VCDIFF inst section length:   5
VCDIFF addr section length:   0

D:\>

lol.patch created with xdelta3 3.0.8, without secondary compressor

Probably a bug inside printhdr, in newer xdelta3 versions.

zonque commented 7 years ago

Ah, so it's just the output of printhdr that is wrong?

mgrinzPlayer commented 7 years ago

Yeah, I think printhdr command is broken in current version.

By the way, if you create a patch with secondary compression (e.g. lzma), VCDIFF header indicator should be like this one:

VCDIFF header indicator:      VCD_SECONDARY VCD_APPHEADER

So, if there's no VCD_SECONDARY, there's no lzma.