Differences on large files fail

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Create a patch file of two 3.7Gb WIM files
2. Process generates a patch file of about 33Mb, then produces the errors
attached.
3.

What is the expected output? What do you see instead?
Errors in attached file.

What version of the product are you using? On what operating system?
Xdelta-3.0o on Windows 2003.

Please provide any additional information below.
Also tried increasing source window size and nominating an unlimited input
size.

Original issue reported on code.google.com by craig.ma...@chh.co.nz on 15 Feb 2007 at 3:40

Attachments:

• results.txt

[GoogleCodeExporter]

Sorry, I couldn't learn much from your attachment.

I would like to help though. 

The messages are just warnings, saying two things: (1) that on a
per-default-window-size level, you have many windows w/o good compression (try 
"-I 0"
to avoid the instrction-buffer overflow issues) (2) some windows have no source
copies (try increasing -B to greater than 64MB and see if that helps).  If the
compression command exits non-zero, please let me know.

Original comment by josh.mac...@gmail.com on 15 Feb 2007 at 10:06

• Changed state: Started

[GoogleCodeExporter]

Also, trying increasing the -W flag to at least 1MB. That's the input window 
size,
wherease -B is the source-window size. -B greatly affects compression 
performance:
the  question is, how great is the difference between absolute-position of the
source- and target- data.  The default for -B is 64MB, so if your data moves 
more
than 32MB in either direction, the compressor will fail to see it.

Xdelta actually allocates buffer space for the source window. A TODO item is to 
use
mmap() for this space. 

Original comment by josh.mac...@gmail.com on 15 Feb 2007 at 10:47

[GoogleCodeExporter]

Apologies, you're right, the operation completed successfully and the patch 
generated
is completely usable. I'm officially a pratt.

I had already tried -I 0, but I still get the instruction buffer overflow 
errors.
Increasing the -W and -B options didn't make the other warnings go away either.

So, to answer the question in your post on Jan 28th, yes we are using xdelta3 
with
files larger than 1Gb and it's working well.

I'm off to try boosting the compression and see how much we can scrunch the 
patch
down. Thank-you for an extremely useful tool that is going to hugely improve our
ability to distribute updates to our build images.

Original comment by craig.ma...@chh.co.nz on 15 Feb 2007 at 9:32

[GoogleCodeExporter]

Great!

There is a bug in the "-I 0" setting code, fixed in current SVN.

The "no source copies" warning is perhaps excessive, maybe I'll make both of 
those be
verbose (-v). They happen in places where the data is completely changed, but 
I've
seen cases where raising -W can avoid those warnings simply because the window 
became
large enough to span another copy.

Original comment by josh.mac...@gmail.com on 16 Feb 2007 at 12:20

• Changed state: Verified

[GoogleCodeExporter]

Fixed in 3.0p

http://en.wikipedia.org/wiki/WIM_image_format

Original comment by josh.mac...@gmail.com on 18 Feb 2007 at 11:48

[GoogleCodeExporter]

Original comment by josh.mac...@gmail.com on 28 Oct 2009 at 3:44

• Changed state: Fixed