When importing a large NZB file, sab fails to import all of it

[pet0ruk]

On the latest version, Windows x64. I noticed when importing a large NZB file (in this case, 6MB and about 76,000 lines of text), that sab simply doesn't import all of it. In this example the download was supposed to be around 65GB but sab only reported it as 35GB and would download it with many files missing.

Splitting the NZB file using https://github.com/samicemalone/nzbsplit would yield a larger and larger download in total. So basically, splitting it in two you'd get a total of maybe 44GB able to download, splitting it into 5 you might get up to 50GB, etc.

A friend was able to reproduce this on his local install of sabnzbd. I cannot reproduce the same problem with nzbget, if I add the same file with the same news servers backing it, it downloads the entire thing just fine.

The result was the same when adding the NZB through the watched folder or through the interface.

[Safihre]

Can you send me the NZB at safihre@sabnzbd.org?

[pet0ruk]

Done!

[Safihre]

This is not a bug, it is because this NZB contains many files with the identical filename. By default SABnzbd will not add files if they occur with the exact same filename twice. This is because many search websites like nzbindex or binsearch often mis-index things and add files twice to an NZB. However, nowadays with obfuscation this can go wrong in season packs like what you send me. You can disable this by going to Config > Specials and turn on allow_duplicate_files, then all files will be added.

[pet0ruk]

Well - lesson learned - thanks. Perhaps it's worth adding some sort of notification or something when this happens beyond a certain threshold (like if sab removes more than 5% or 10% of the files or something).

The reason I mention this is that I've been using sab for many years now, and I've nearly always had trouble with season packs so I mostly try to avoid them. But recently I decided to try and dig to get to the bottom of why this was happening (because all the files I had seemed to be complete, but I just didn't have all the files), and I still didn't find any references to this option or its effects, but maybe I was just Googling the wrong things. It's pretty easy to do what I did and assume it might be a bug in sab when you throw the nzb in another client and it works as expected.

Thanks for your work!

[Safihre]

Yeah, you are right, I did not consider this scenario, only those faulty NZBs. Maybe it's worth a warning if many are detected.

[Safihre]

Added the warning in case more than 10 files were discarded 👍

[pet0ruk]

Awesome thanks! :)