Open psvajaz opened 1 year ago
Hi!
1、Unable to obtain the file list of the partitioned compressed package using the BitArchiveReader class
Did you open the archive using BitFormat::Split
or with the specific archive format?
In the first case, BitArchiveReader
will only report one item: the whole archive; this is the expected behavior.
On the other hand, if you used the specific archive format, then it's a bug! Are you trying to open more than 999 volumes also in this case?
2、When split compressing by package , if the number of packages exceeds 999, an error will be reported
I can reproduce the error. I'll fix this bug as soon as possible!
I used BitFormat:: Split for preview and obtained content with only one item. In this situation, how should I continue previewing the content of this item?
I think that in this situation, Bit7z should automatically help me obtain the content of this compressed package. Do you think this is more reasonable
I used BitFormat:: Split for preview and obtained content with only one item. In this situation, how should I continue previewing the content of this item?
Unfortunately, it's not possible when opening the archive using BitFormat::Split
.
I think that in this situation, Bit7z should automatically help me obtain the content of this compressed package. Do you think this is more reasonable
Bit7z does support automatically reading the content of the compressed package in a multi-volume archive, but you have to specify the compression format, rather than BitFormat::Split
.
For example:
BitArchiveReader reader{ lib, "archive.7z.001", BitFormat::SevenZip };
// reader will directly read inside the whole 7z archive
Thank you
https://www.7-zip.org/a/7z2201-extra.7z I used this compressed package, decompressed it, and recompressed it using the official 7z tool. The compressed file type is 7z, and the partition size is 1kb. After compression, I obtained 1049 files for each partition. When using BitInputArchive to load the compressed package and debugging the bit7z source code, it was found that the while loop in CMultiVolumeInStream:: CMultiVolumeInStream loops to volume_index=510, the CFileInStream:: open method will fail to open the file, and the test environment is x86
Execution After _setmaxstdio (8192), when previewing compressed packets with more than 999 volumes, normal operation is possible There are two issues with the test now:
During these two days of debugging, it was found that the Stream could not be released normally, causing IO abnormalities
I used this compressed package, decompressed it, and recompressed it using the official 7z tool. The compressed file type is 7z, and the partition size is 1kb. After compression, I obtained 1049 files for each partition. When using BitInputArchive to load the compressed package and debugging the bit7z source code, it was found that the while loop in CMultiVolumeInStream:: CMultiVolumeInStream loops to volume_index=510, the CFileInStream:: open method will fail to open the file, and the test environment is x86
Execution After _setmaxstdio (8192), when previewing compressed packets with more than 999 volumes, normal operation is possible
Thanks for the debugging!
Yeah, I actually noticed this issue while fixing the bug you initially reported here.
The main problem is that bit7z keeps all the volume streams open until the compression/extraction operation finishes. This is actually the same behavior that 7-zip had before the latest v23.01 (still not supported by bit7z, by the way, but I'll work on it). On Windows, the default limit is 512 opened file handles, hence the error when trying to operate on archives with many volumes.
Yesterday, I pushed two commits to the develop
branch (https://github.com/rikyoz/bit7z/commit/4121bff0210081c960a54a37ca43256dd493844e and https://github.com/rikyoz/bit7z/commit/8a0c1c93c11c0c32ecaf863da8243fa03149959e) that fix the problem, at least when compressing.
This is just a temporary workaround that uses _setmaxstdio
, as you suggested.
I'll also push the same fix for CMultiVolumeInStream
as soon as possible.
Ideally, the library should keep fewer streams opened, but this would require more code changes, so a cleaner fix will probably come after the v4 stable release.
When compressing in separate volumes, a maximum of 999 split packages can be generated
Isn't this the same issue you initially reported? If so, I already fixed it on develop
(https://github.com/rikyoz/bit7z/commit/260ebe06acbb6f1efee53e5caa96c80cf996ec90).
When decompressing a compressed package with more than 999 volumes, it cannot be decompressed properly
I'll check this, thanks!
During these two days of debugging, it was found that the Stream could not be released normally, causing IO abnormalities
Could you give more info about this issue?
Quick update: I just pushed a commit to develop
that should fix the opened files limit issue also for CMultiVolumeInStream
(e787c3de9a56b9c3c4a4a63051629b26f5afa5a1).
OK, Thanks. I will Test
compress and decompress 999 split packages the problem still exists.
preview is ok
Unfortunately, this time I cannot reproduce the problem. In my tests, bit7z handles archives with more than 2000 volumes just fine.
Can you tell me if you can call the code for decompression and compression methods and let me see if it's my coding problem
Compression:
BitFileCompressor compressor{ lib, BitFormat::SevenZip };
/* Setting the volume size as 1/2048 of the uncompressed file size;
obviously, we will have less than 2048 volumes due to compression,
but they will still be more than 2000, at least in my case! */
compressor.setVolumeSize( file_size / 2048 );
compressor.compressFile( "path/to/file.exe", "multi_volume.7z" );
Extraction:
BitArchiveReader reader{ lib, "multi_volume.7z.001", BitFormat::SevenZip };
// Output folder which will contain the original unsplit and uncompressed file.
reader.extractTo( "output/" );
alternatively
BitFileExtractor extractor{ lib, BitFormat::SevenZip };
extractor.extract( "multi_volume.7z.001", "output/" );
This error will be reported when the number of questionnaires exceeds 8188
bit7z version
4.0.x RC
Compilation options
BIT7Z_AUTO_FORMAT, BIT7Z_USE_NATIVE_STRING
7-zip version
v22.01
7-zip shared library used
7z.dll / 7z.so
Compilers
MSVC
Compiler versions
MSVC2017
Architecture
x86_64, x86
Operating system
Windows
Operating system versions
Windows 11
Bug description
1、Unable to obtain the file list of the partitioned compressed package using the BitArchiveReader class 2、When split compressing by package , if the number of packages exceeds 999, an error will be reported
Steps to reproduce
No response
Expected behavior
No response
Relevant compilation output
No response
Code of Conduct