guzba / zippy

Pure Nim implementation of deflate, zlib, gzip and zip.
MIT License
246 stars 29 forks source link

wrong number of entries for zip files #40

Closed guaracy closed 2 years ago

guaracy commented 2 years ago

zip file from: https://github.com/google/material-design-icons/archive/refs/heads/master.zip

import zippy/ziparchives, zip/zipfiles

# -----------------------------------

let filename = "material-design-icons-master.zip"
let z = openZipArchive(filename)
var ix : int64
echo "zippy"
for f in z.walkFiles:
    inc(ix)
echo ix," entries"

# -----------------------------------

ix = 0
var z1: zipfiles.ZipArchive
discard z1.open(filename)
echo "zip"
for f in z1.walkFiles:
    inc(ix)
echo ix," entries"

results

zippy
48629 entries
zip
952432 entries

environment

nim  1.6.2 stable
Linux Mint 64
guzba commented 2 years ago

I took a look and I think Zippy is correct here. It looks like zipfiles walkFiles includes all entries (files and directories). It calls https://libzip.org/documentation/zip_get_num_files.html which is deprecated and calls through to https://libzip.org/documentation/zip_get_num_entries.html

This is wrong IMO, as it does not match Nim std/os walkFiles which only yields files https://github.com/nim-lang/Nim/blob/35c812fda1ae7f1d183d3021793438fb2099c329/lib/pure/os.nim#L2152

guzba commented 2 years ago

The actual issue here is this is not a ZIP archive, it is a ZIP64 archive. They both share the same file extension but I need to detect and error out on ZIP64 it appears.

guzba commented 2 years ago

Release 0.9.6 supports more ZIP64, I now get the correct number of entries etc. That archive should probably work, but it's 100k files and will take forever so I'll leave that to you to confirm.