"get oldPath type ERROR!" on Windows XP

[pkr-sadx]

Hello, thank you for making this extremely useful tool.

I've been trying to compile a Windows XP compatible version of hpatchz.exe for a retro project I'm working on. It compiles successfully in VS2022 after changing platform toolset to v141_xp for all projects and setting subsystem to CONSOLE. The program itself runs on Windows XP, however it fails to apply the patch with get oldPath type ERROR!. The same executable with the same command line arguments applies the patch successfully on Windows 7 or later. Do you know what causes this difference and what changes to the code are required to get XP compatibility?

I realize this is a pretty uncommon issue, so I am not requesting any support for XP, but if there's a way to get it working on XP I'd appreciate it if you could let me know. Thanks!

EDIT: Forgot to say, the patch I'm trying to apply is a folder patch. Individual file patches appear to be working fine.

[mgrinzPlayer]

Which source? tag/v4.6.7 v4.6.8 or latest tag/v4.6.9

I can make tests

[sisong]

Guessing that _wstat64() are not winxp compatible?
now, I have no winxp & vc for debug this.

[pkr-sadx]

Hi, thank you for the response. I tried v4.6.7 and the latest commit, both with the same result. I'll be happy to test on a Windows XP VM and on real hardware.

[mgrinzPlayer]

@pkr-sadx I compiled it with MSYS2 + mingw-w64 + patched HDiffPatch makefile and patched zstd threading.c/threading.h It successfully patched folder on my WinXPSP3. Note: it was source v.4.6.3 and packages from that time. I will try with latest commits (and latest packages).

// Yes, you can run latest commit (MSYS2 + patches mentioned before) on 32bit WindowsXP.

[sisong]

@mgrinzPlayer The cmdline you compile on mingw，it call stat() default, which should does not support file larger than 4GB.

@pkr-sadx recode _hpatch_getPathStat_noEndDirSeparator() to:

    struct stat  s;
    int          rt;
    assert(out_type!=0);
    memset(&s,0,sizeof(s));
    rt = stat(path_utf8,&s); //only for test,not support file larger than 4GB
    ... <following the original code> ...

[pkr-sadx]

@sisong I changed file_for_patch.c (latest commit) like this and recompiled.

Unfortunately I'm still getting the same error on XP. I've also tried changing _IS_USED_WIN32_UTF8_WAPI in file_for_patch.h to 0 both with and without the above change, no luck.

@mgrinzPlayer I've been trying with VS2022 until now, will try with your setup, thanks.

[sisong]

I searched some infos of winxp compatibility ?stat?(); For example： https://github.com/gabime/spdlog/issues/280 https://stackoverflow.com/questions/32452777/visual-c-2015-express-stat-not-working-on-windows-xp https://learn.microsoft.com/en-us/answers/questions/168279/tstati64-function-is-not-supported-on-this-system

It might seem like the safest way to compatible winXP is to use GetFileAttributesEx().

[mgrinzPlayer]

I made some patches to the makefile, so it is using stat64. Let me remove my other custom patches and post the needed patches here.

EDIT: patch for makefile:

Index: Makefile
===================================================================
--- Makefile
+++ Makefile
@@ -212,7 +212,8 @@
     -D_IS_NEED_ALL_CompressPlugin=0 \
     -D_IS_NEED_DEFAULT_CompressPlugin=0 \
     -D_IS_NEED_ALL_ChecksumPlugin=0 \
-    -D_IS_NEED_DEFAULT_ChecksumPlugin=0 
+    -D_IS_NEED_DEFAULT_ChecksumPlugin=0 \
+    -D_IS_USED_WIN32_UTF8_WAPI=1
 ifeq ($(RISCV32),0)
 else
   DEF_FLAGS += -D_IS_NO_ATOMIC_U64=1
@@ -219,7 +220,7 @@
 endif
 ifeq ($(M32),0)
 else
-  DEF_FLAGS += -m32
+  DEF_FLAGS += -m32 -Wl,--large-address-aware
 endif
 ifeq ($(MINS),0)
 else
@@ -310,7 +311,7 @@
     -D_IS_USED_CPP11THREAD=1
 endif

-PATCH_LINK := 
+PATCH_LINK := -municode
 ifeq ($(ZLIB),2)
   PATCH_LINK += -lz            # link zlib
 endif
@@ -343,7 +344,7 @@
 ifeq ($(STATIC_CPP),0)
   DIFF_LINK += -lstdc++
 else
-  DIFF_LINK += -static-libstdc++
+  DIFF_LINK += -static
 endif

 CFLAGS   += $(DEF_FLAGS) 

patch for zstd

Index: threading.c
===================================================================
--- threading.c
+++ threading.c
@@ -20,7 +20,7 @@
 /* create fake symbol to avoid empty translation unit warning */
 int g_ZSTD_threading_useless_symbol;

-#if defined(ZSTD_MULTITHREAD) && defined(_WIN32)
+#if defined(ZSTD_MULTITHREAD) && defined(_WIN32) && !defined(__MINGW32__)

 /**
  * Windows minimalist Pthread Wrapper
Index: threading.h
===================================================================
--- threading.h
+++ threading.h
@@ -20,7 +20,7 @@
 extern "C" {
 #endif

-#if defined(ZSTD_MULTITHREAD) && defined(_WIN32)
+#if defined(ZSTD_MULTITHREAD) && defined(_WIN32) && !defined(__MINGW32__)

 /**
  * Windows minimalist Pthread Wrapper

MSYSTEM=MINGW32

make.exe -B ZLIB=1 BZIP2=1 STATIC_CPP=1 MINS=1 M32=1

MSYSTEM=MINGW64

make.exe -B ZLIB=1 BZIP2=1 STATIC_CPP=1 MINS=1

Binaries: MSYS2 MINGW-W64.zip

hdiffz imports: msvcrt.dll _filelengthi64 ord:261 rva2iat: 001C73A8 _wstat64 ord:1045 rva2iat: 001C73F4

hpatchz imports those too: msvcrt.dll _filelengthi64 ord:261 rva2iat: 00065278 _wstat64 ord:1045 rva2iat: 000652B0

[pkr-sadx]

@mgrinzPlayer Thank you for sharing the patches and the binaries. I was able to confirm that the i686 binary works on Windows XP SP3 on a VM, and on Windows XP Embedded on hardware.

Many thanks for your work and insights!

[mgrinzPlayer]

Good. I'm glad it worked for you.

There's one tiny problem, currently, 32bit hdiffz (original release from git and my msys2-mingw-w64 build) can not allocate more than 2GB. I ensured that LAA (large address aware) is set on both EXE files.

If I'm not wrong, if you process 5GB old and 5GB new file and -p-1 -s-32, it should take about (5368709120 16 / 32) bytes of memory. Which is about 2.5GB. Formula is O(oldFileSize16/matchBlockSize + ...)

For me, 32bit hdiffz didn't allocate more than 2GB. Under WinXP32bit, with /3GB switch, memory limit is 3GB. Under newer Win64bit, memory limit is 4GB - info link

[sisong]

@pkr-sadx I update some test code for WinXP, swap stat() to GetFileAttributesEx():
https://github.com/sisong/HDiffPatch/tree/dev-forWinXP

[pkr-sadx]

Hi @sisong

Thanks for making this. I switched to the new branch and compiled hpatchz for v141_xp. Folder patch applied successfully on Windows XP.