search

emmercm / igir

🕹 A zero-setup ROM collection manager that sorts, filters, extracts or archives, patches, and reports on collections of any size on any OS.
https://igir.io/
GNU General Public License v3.0
364 stars 18 forks source link

failed to create zip: TypeError [ERR_INVALID_ARG_TYPE]: The "start" argument must be of type number. Received null #991

Closed tnvu closed 2 months ago

tnvu commented 7 months ago

Paste the command

./igir move zip test -d dat/Nintendo\ -\ Wii\ U\ (Digital)\ (CDN)\ (20231116-082151).dat -i input -o output -vvv

Describe the bug

On some archives, it prints out the error message

failed to create zip: TypeError [ERR_INVALID_ARG_TYPE]: The "start" argument must be of type number. Received null

Expected behavior

zip file should be created successfully

Debug logs

igir.log

DAT(s) used

https://datomatic.no-intro.org/index.php?page=download&op=dat&s=107

igir version

2.5.0

Node.js version

N/A

Operating system

Linux

Additional context

This is also causing the igir process to hang at the end of processing.

emmercm commented 7 months ago

@tnvu was this with a pre-compiled version of igir?

tnvu commented 7 months ago

yes, v2.5.0 pre-compiled

emmercm commented 7 months ago

There's probably a problem with error handling in igir somewhere, but I can't seem to reproduce this exact issue.

Could you test your input zip file, like this:

unzip -t "input/Amazing Spider-Man, The - Ultimate Edition (USA) (En,Fr,Es).zip"
tnvu commented 7 months ago

"No errors detected in compressed data of Amazing Spider-Man, The - Ultimate Edition (USA) (En,Fr,Es).zip."

So I did a few tests:

1) Unzipped the original archive and ran igir on the extracted files = SUCCESS 2) Fed the newly created archive back into igir = SUCCESS

So there is something funky about the weird about the original archive and how igir is handling it... but only when it's in the archived format.

I then converted the zip file produced by igir into a torrentzip and it has the same MD5 as the original (problematic) file. So if you have that particular file already, convert it to TorrentZip and you should be able to reproduce the problem.

emmercm commented 7 months ago

@tnvu I'll work on doing that on my end.

Meanwhile, could you test this PR and tell me if it helps you, instructions are in a comment: https://github.com/emmercm/igir/pull/994

tnvu commented 7 months ago

This worked for me in that it successfully exits (doesn't hang anymore) and cleans up the temporary files. It should resolve #981

It still has the start argument error though

[20:35:26.513] ERROR: CandidateWriter: Nintendo - Wii U (Digital) (CDN): Amazing Spider-Man, The - Ultimate Edition (USA) (En,Fr,Es): output/Amazing Spider-Man, The - Ultimate Edition (USA) (En,Fr,Es).zip: failed to create zip: failed to create patched read stream for input/Amazing Spider-Man, The - Ultimate Edition (USA) (En,Fr,Es).zip|00000019.h3: Error: failed to create stream for input/Amazing Spider-Man, The - Ultimate Edition (USA) (En,Fr,Es).zip|00000019.h3: TypeError [ERR_INVALID_ARG_TYPE]: The "start" argument must be of type number. Received null

emmercm commented 7 months ago

Thank you for testing! That failed to create stream for is the key piece here. There's something wrong with https://www.npmjs.com/package/unzipper that I'll likely need to report to them.

In the meantime, I'll work on getting this change in so at least igir doesn't hang forever.

keywal commented 3 months ago

Hi @emmercm Christian,

Just wanted to report that i have this error today too from the pre-compiled version.

I added some new DAT files into the mix and assumed it was those that caused the issue but having read the above its likely that igir has been asked to process a zip file it ignored before and now will not continue. I've removed the DAT files and its processing normally now.

Unfortunately the output of the error didnt give much in the way of a path to the file with the issue so i couldnt resolve manually.

emmercm commented 3 months ago

@tnvu so that I can reproduce, what TorrentZip tool are you using and what options did you provide to it?

@keywal please enable trace logging with the -vvv option and upload the log file.

tnvu commented 3 months ago

https://github.com/0-wiz-0/trrntzip

version 1.3

no options provided

emmercm commented 3 months ago

I believe https://github.com/ZJONSSON/node-unzipper/issues/324 is the issue here.

I can put in a fix to stop these crashes, but igir won't be able to fully read these files until the library makes a fix.

github-actions[bot] commented 1 month ago

:lock: Inactive issue lock

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Comment generated by the GitHub Lock Issues workflow.