bzip2 files more then 4gb possibly stuck decoding

[GoogleCodeExporter]

What steps will reproduce the problem?
1. xdelta3 -I 0 -B 4294967 -d -f -s source_file(archive.tar.bz2) patch (new 
file)
2. archive.tar.bz2 ~2GB (when it is unpacket, it's over 6GB)

What is the expected output? What do you see instead?
nothing, this works endless, i have waited over 24 hours.

if i decrease -B size, then i have error:
xdelta3: externally compressed input: bzip2 -dc < 2.0.04.61.tar.bz2
xdelta3: target window checksum mismatch: XD3_INVALID_INPUT

What version of the product are you using? On what operating system?
3.0.0(without letters), last from official site

Please provide any additional information below.

Original issue reported on code.google.com by dolgov...@gmail.com on 31 Aug 2011 at 2:36

• Merged into: #132

[GoogleCodeExporter]

Several things may be happening to explain this. If you still have the files 
and want to track it down, or it's possible to download them somewhere, I'm 
happy to help.

First, you're using an extremely small window size (-B around 4MB) for such 
large inputs.  I suspect you are trying to use a 4GB source window, instead?  
Using a small buffer alone should not cause an endless loop, but it will hurt 
performance which can cause slowness.  If you set "-vv" flags it will print a 
progress report for each input window, which should give you an idea where it's 
stuck.

Second, be aware that when either the source or the target inputs are 
compressed by another program (bzip2 in this case), xdelta3 runs an external 
program to decompress the data, which could be a source of problems.  I usually 
test with gzip and not on such large compress inputs, so it's conceivable that 
there is some kind of buffering problem related to bzip2. So, to track this 
down I recommend decompressing the inputs first and seeing if xdelta3 then runs 
correctly (with a larger -B setting, too).

Original comment by josh.mac...@gmail.com on 18 Jun 2012 at 12:46

• Changed title: bzip2 files more then 4gb possibly stuck decoding
• Changed state: Started

[GoogleCodeExporter]

Actually, this looks like a duplicate of 132 related to bzip2 buffering

Original comment by josh.mac...@gmail.com on 18 Jun 2012 at 12:47

• Changed state: Duplicate