-chunk doesn't work with encryption

[busyammonia]

I want to create an encrypted multipart archive, but have trouble in doing so. It works fine without encryption.

[f@pc:~/Documents]$ zpaqfranz a "default_????.zpaq" ./kovcheg -m1 -chunk 1G -key
zpaqfranz v59.2q-JIT-L(2024-02-23)
franz:-method                                   1
franz:-chunk                        1.000.000.000

Please do NOT start with '-' (minus) 
Enter password :******************************************
franz:-key                               (hidden)
Creating default_0001.zpaq at offset 32 + 0                                                               
zpaqfranz error: 37992: cannot read salt
8.631 seconds (000:00:08) (with errors)

[fcorbelli]

It is a bug, present only in the *nix version (on Windows it does works) ... work in progress...

[fcorbelli]

Please check the attached pre-release 59_3a.zip

And yes, the fix is a...fflush 😄

Let me know if OK Thank you

[busyammonia]

It does seem to work now, thank you.

[f@pc:~/Documents]$ /tmp/zpaqfranz a "test_????.zpaq" kovcheg/ -m1 -key -chunk 490M
zpaqfranz v59.3a-JIT-L(2024-02-25)
franz:-method                                   1

Please do NOT start with '-' (minus) 
Enter password :******************************************
franz:-key                               (hidden)
franz:-chunk                          490.000.000
Creating test_0001.zpaq at offset 32 + 0                                                                  
Add 2024-02-26 10:03:39    19.273      4.707.961.405 (   4.38 GB) 12T (5.076 dirs)
24.349 +added, 0 -removed.                                                                                
INFO: The last chunk is: test_0005.zpaq

32 + (4.707.961.405 -> 3.136.879.839 -> [CKS #5] 1.614.066.183) = 1.614.066.215 [TOT] @ 89.91 MB/s
49.938 seconds (000:00:49) (all OK)

[f@pc:~/Documents]$ zpaqfranz t "test_????.zpaq"
zpaqfranz v59.2q-JIT-L(2024-02-23)
Archive seems encrypted (or corrupted)
Enter password :******************************************

test_????.zpaq: 
1 versions, 24.349 files, 2.104.078.887 bytes (1.96 GB)
To be checked 4.707.961.405 in 19.273 files (12 threads)
7.15 stage time      12.78 no error detected (RAM ~192.78 MB), try CRC-32 (if any)
Checking            20.966 blocks with CRC-32 (4.695.889.249 not-0 bytes)
Block 00020K          4.32 GB
CRC-32 time           0.63s
Blocks       4.695.889.249 (      20.966)
Zeros           12.072.156 (          40) 0.011000 s
Total        4.707.961.405 speed 7.508.710.374/sec (6.99 GB/s)
GOOD            : 00019273 of 00019273 (stored=decompressed)
VERDICT         : OK                   (CRC-32 stored vs decompressed)
13.405 seconds (000:00:13) (all OK)

[fcorbelli]

This is the fix... the fflush With encryption zpaq write a 32 bytes-long "prequel" on the archive, the "salt" Then, in another position of the source code, read back (from the filesystem, with a fopen/fread on the archive) those 32 bytes When the archive ALREADY exists (aka: updating), no problem. But when archive (the very first "chunk") does NOT (aka: creating), on some non-Windows systems the write (of the seed, the very first 32 bytes) is keeped in some operating system buffer, cache or whatever. Therefore, when zpaq read back the salt... get nothing, because the file (in the filesystem) is 0 bytes long, and not 32 as expected => fflushing enforce to transfer the 32 bytes to the file, making a 32-bytes-long file, that can be readed back

if (off==0 && myfwrite(salt, 1, 32, fp)!=32)
    ioerr(thefilename.c_str());
fflush(fp); /// unix fix

Be careful with fixed-sized-multipart, it is a "cutting edge" feature (required a LOT of spaghetti-code)

Thanks for your report

[polybluez]

Can you please make a new release with this fix so that Linux distros would upgrade it?

[fcorbelli]

The source code already provides for the use of -chunk with encryption (the only function that cannot be used is -index, and I don't think it will be in the future) If you are referring to the packages in the various distributions it is a very complex matter, it is not so easy to update them (it is not up to me). I will try to contact the maintainers