Rare failure with xdelta3-3.0.11-i686.exe

[juanitogan]

I believe I have a reliable report on a possible bug from one of my users. I, unfortunately, cannot replicate it myself, so this report may not be of much help. Sorry about that. Still, I thought I should report it just in case someone else can replicate it. I am not close to this user, personally or distance, so testing may not be a reasonable expectation.

The user sent me the following screenshot of the failure and reported it working on several machines except for one, his primary machine, of course. All I know about that one right now is it runs Windows 7, 64-bit. The user did several binary comparisons of the file in question versus copies he sent me, so I am pretty confident this has something to do with xdelta3 and the peculiarities of his machine.

target window checksum mismatch: XD3_INVALID_INPUT

What you see in the screenshot is my patcher batch file first trying the USA patch for Mooncrat.rbx, and then the EU patch. In this case, the first patch fails quickly, as expected. The second patch runs partway and then fails. The second patch normally runs fine in this particular case. Here is a link to a zip of the relevant files (Mooncrat.rbx and mooncrat_EU.patch):

https://1drv.ms/u/s!AkReq58OpwI_iNxfD6jiOlHR-wJgCw

These are the commands the batch would be running in this screenshot (the second line is the relevant one):

xdelta3 -dfv -s "%localpath%\mooncrat.RBX" mooncrat.patch "%localpath%\mooncrat.WIN7FIX"
xdelta3 -dfv -s "%localpath%\mooncrat.RBX" mooncrat_EU.patch "%localpath%\mooncrat.WIN7FIX"

xdelta3.exe is renamed from xdelta3-3.0.11-i686.exe. It is patching a 32-bit program so testing the 64-bit version of Xdelta3 was not relevant since I need this to run on 32-bit OSs.

If you want more context, this is part of a patcher I make for 3D Ultra MiniGolf Deluxe which is found here:

https://github.com/juanitogan/rbxit/wiki/3D-Ultra-MiniGolf-Series

Thank you for this awesome delta tool. It is just what I need for bulky data differences versus the more common tools.

Matt

[juanitogan]

I should also note that several files process just fine before this one is hit and fails.

[hurda]

Patching it on a Win7 64bit machine using both 32bit and 64bit executables works for me.

>xdelta3-3.0.11-i686.exe -dfv -s Mooncrat.rbx mooncrat_EU.patch foobar.rbx
xdelta3: source Mooncrat.rbx source size 17.2 MiB [18022462] blksize 64.0 MiB window 64.0 MiB
xdelta3: 0: in 4.02 MiB: out 8.00 MiB: total in 4.02 MiB: out 8.00 MiB: 1.1 sec
xdelta3: 1: in 28.0 B: out 8.00 MiB: total in 4.02 MiB: out 16.0 MiB: 47 ms
xdelta3: 2: in 28.0 B: out 4.64 MiB: total in 4.02 MiB: out 20.6 MiB: 31 ms
xdelta3: finished in 1.2 sec; input 4212191 output 21643417 bytes (513.83%)

There are several closed issues here with the same error which came down to the source-file not being the same as the patch requires. Maybe add a checksum-check before trying to patch, just to make sure?

[jmacd]

I have many times considered doing what you suggest -- a checksum check on the source file. Xdelta-1.x did that, but it takes a second pass over the source file, which xdelta-3.x is explicitly designed not to do, as this enables stream-processing of indefinite-length data. The fact is, the error being generated is due to a checksum, but it's an output checksum, not an input checksum. So I decided to call this working-as-intended, the error message does suggest the most frequent cause is a wrong source file. I recommend applications that manage xdelta from another app verify checksums themselves.

On Mon, Jan 30, 2017 at 2:55 AM, hurda notifications@github.com wrote:

Patching it on a Win7 64bit machine using both 32bit and 64bit executables works for me.

xdelta3-3.0.11-i686.exe -dfv -s Mooncrat.rbx mooncrat_EU.patch foobar.rbx xdelta3: source Mooncrat.rbx source size 17.2 MiB [18022462] blksize 64.0 MiB window 64.0 MiB xdelta3: 0: in 4.02 MiB: out 8.00 MiB: total in 4.02 MiB: out 8.00 MiB: 1.1 sec xdelta3: 1: in 28.0 B: out 8.00 MiB: total in 4.02 MiB: out 16.0 MiB: 47 ms xdelta3: 2: in 28.0 B: out 4.64 MiB: total in 4.02 MiB: out 20.6 MiB: 31 ms xdelta3: finished in 1.2 sec; input 4212191 output 21643417 bytes (513.83%)

There are several closed issues here with the same error which came down to the source-file not being the same as the patch requires. Maybe add a checksum-check before trying to patch, just to make sure?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jmacd/xdelta/issues/227#issuecomment-276033118, or mute the thread https://github.com/notifications/unsubscribe-auth/ADdiiXDmupctG1T42U2olfCEe-8iy1NBks5rXcGkgaJpZM4LnvOR .

[juanitogan]

Thank you, @hurda , for testing it. I am not surprised with your result given that I have had only one valid report of this so far. I have tested this a on a few platforms, including Win7 64-bit, myself and cannot reproduce it.

I assure everyone, I have checked all files in question with binary comparisons multiple times and so has the user. It works for the user on all machines except one. It does not appear to be a source file issue. That is the obvious culprit, but it really does not appear to be that.

Also, yes, I have considered adding a checksum to the patching process... but I have not gone there yet ( @jmacd , I think @hurda meant that comment for me and my batch file and not for xdelta -- I agree with your reasons for leaving it out of xdelta). The errors and inline checksums reported by xdelta are usually sufficient to tell what's what when a user has an issue. In this case, however, something else appears to be up that is specific to a small set of machines.

Anyhow, I know that this likely cannot be fixed without more intel. I figure all I can really do is document what I have here in case something pops up related to it down the road... or maybe we get lucky and someone here can reproduce it.

[juanitogan]

BTW, I expect I might be able to get the user to send me a system information report, of something of the sort, if you like.

[hurda]

Has the user tried checking the machine with memtest? Once I had a case of unexplainable crashes and errors of a single program on a single machine turning out to be dying RAM-modules. Maybe that's the case here.

I have many times considered doing what you suggest -- a checksum check on the source file.

I meant that check to be in the patcher, which would also allow the patcher to use the appropriate patch for a file instead of brute-forcing it.