Closed kokutoukiritsugu closed 1 month ago
Is it an archivefile issue or is it a py7zr issue? Can you check if you get the same issue when using py7zr on it's own?
py7zr no problem
with py7zr.SevenZipFile(full_file_path) as archive1:
for fname, bio in archive1.readall().items():
print(f'{fname}: {bio.read(21)}')
asdf_10.pdf: b'b\x14#eb\x00\x9e\x01\x00\x00\x00\x01somebyte'
asdf_3.pdf: b'b\x14#eW\x00\xa9\x01\x00\x00\x00\x01somebyte'
asdf_4.pdf: b'b\x14#el\x00\x94\x01\x00\x00\x00\x01somebyte'
asdf_5.pdf: b'b\x14#ek\x00\x95\x01\x00\x00\x00\x01somebyte'
asdf_6.pdf: b'b\x14#eh\x00\x98\x01\x00\x00\x00\x01somebyte'
asdf_7.pdf: b'b\x14#e_\x00\xa1\x01\x00\x00\x00\x01somebyte'
asdf_8.pdf: b'b\x14#eq\x00\x8f\x01\x00\x00\x00\x01somebyte'
asdf_9.pdf: b'b\x14#ep\x00\x90\x01\x00\x00\x00\x01somebyte'
If you can give me steps to reproduce this, I can probably look into fixing this
just use 7-Zip compress some file check solid
use read_bytes(am.name)[12:21]
I've added solid 7z files to test_data (https://github.com/Ravencentric/archivefile/commit/8cccb956e9b1a10250e5ef74be26cf6523bf7c63) and added read tests (https://github.com/Ravencentric/archivefile/commit/2e56f58cb932628c2335ce797dad7370a2d2445a). As you can see, the tests pass without issues and I cannot reproduce this on my end. Unless you give me concrete reproduction steps I cannot help you anymore.
problem 7z file inside this zip file. 3月1_2.zip
I'll take another look
@kokutoukiritsugu I fixed it in https://github.com/Ravencentric/archivefile/pull/3. Would be nice if you could test it and let me know before I merge and release
function ok, but speed slow in a lot of file in 7z...
with archivefile.ArchiveFile(apb, 'r') as archive:
for name in archive.get_names():
if archive.get_member(name).is_file:
check_archive_enc1(apb, name, archive.read_bytes(name)[12:21])
vs
with py7zr.SevenZipFile(apb) as archive:
for name, bio in archive.read().items():
if not name.endswith("/"):
check_archive_enc1(apb, name, bio.read(21)[12:21])
You can do for member in archive.get_members()
there. Being slower than the dedicated library is expected because archive file is a wrapper after all but if you can time it that would be nice to get an idea of how slow it actually is.
i try, use for member...
6.49850606918335 vs 0.5358200073242188
yes, read_bytes not best suitable for warpper
That's slower than I expected. Anyway that's something I'll look into now but not really an immediate goal. I'll close this issue when I merge #3
ok thanks very much !
from time import perf_counter
import archivefile
import py7zr
file = "3月1.7z"
start = perf_counter()
with archivefile.ArchiveFile(file) as archive:
for member in archive.get_members():
if member.is_file:
archive.read_bytes(member)
print(perf_counter() - start)
start = perf_counter()
with py7zr.SevenZipFile(file) as archive:
for name, bio in archive.read().items():
if not name.endswith("/"):
bio.read()
print(perf_counter() - start)
ArchiveFile: 0.013419300004898105
SevenZipFile: 0.007313699999940582
Although it will never beat the underlying library for obvious reasons, think I'm happy with the minor improvements
solid 7z file will get error. non-solid 7z file no problem.