Methods from module zip read only 65535 records from zip file

[sigsergv]

Describe the bug

Methods like zip:foldl or zip:unzip process limited number of records from zip archive. No errors just incomplete list.

To Reproduce

Try to unpack or list_dir of provided file test.zip (it consists of 80000 empty files).

root@debian-experimental-amd64:~# erl
Erlang/OTP 27 [erts-15.0.1] [source] [64-bit] [smp:32:12] [ds:32:12:10] [async-threads:1] [jit:ns]

Eshell V15.0.1 (press Ctrl+G to abort, type help(). for help)
1> zip:foldl(fun(F, G, B, Acc) -> Acc + 1 end, 0, "test.zip").
{ok,65535}
2> 

The same for zip:unzip, in unpacks not all files.

System unzip command reads and unpacks everything:

root@debian-experimental-amd64:~# unzip -t test.zip |wc -l
80004

Expected behavior

Methods from module zip should process all files from provided archive.

Affected versions

27

Additional context

Test file: test.zip Checked on otp-25, otp-27 from debian

[jhogberg]

Hi! This should be resolved by https://github.com/erlang/otp/pull/8537, can you try it out?

[sigsergv]

Hi! Nope, still reproducible:

[sigsergv@webstation:~/projects/erlang/erlang-otp]% ./bootstrap/bin/erl                        
Erlang/OTP 27 [erts-15.0] [source-7d87e7b09c] [64-bit] [smp:32:12] [ds:32:12:10] [async-threads:1] [jit:ns]

Eshell V15.0 (press Ctrl+G to abort, type help(). for help)
1> zip:foldl(fun(F, G, B, Acc) -> Acc + 1 end, 0, "/home/sigsergv/tmp/test-erl-zip/test.zip").
{ok,65535}
2> 

[jhogberg]

Thanks! I don't have the time to dig into this further as most of us are on vacation right now, but we'll revisit this once we're back. :-)

[garazdawi]

The functionality works with #8537 if you don't use the bootstrap emulator.

> ./bin/erl
Erlang/OTP 27 [erts-15.0] [source-1c0f6cb2d2] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Eshell V15.0 (press Ctrl+G to abort, type help(). for help)
1> zip:foldl(fun(F, G, B, Acc) -> Acc + 1 end, 0, "test.zip").
{ok,80002}

[sigsergv]

Checked on latest pull request contents of #8537 and it's working correctly indeed:

[sigsergv@webstation:~/projects/erlang/erlang-otp]% ./bin/erl
Erlang/OTP 27 [erts-15.0] [source-4ca37b747a] [64-bit] [smp:32:12] [ds:32:12:10] [async-threads:1] [jit:ns]

Eshell V15.0 (press Ctrl+G to abort, type help(). for help)
1> zip:foldl(fun(F, G, B, Acc) -> Acc + 1 end, 0, "/home/sigsergv/tmp/test.zip").
{ok,80002}
2> 

[garazdawi]

Fix merged to maint for release in 27.1.