Only the first file is downloaded when selecting multiple files from an RSS feed

Is there already an issue for your problem?

[X] I have checked older issues, open and closed

NZBGet Version

v24.2-stable

Platform

Windows

Environment

Device:
OS version: Windows 10 22H2 64-bit
CPU architecture: 64-bit
Includes libs or tool that apply: 7-Zip 24.07, UnRAR 7.01
Running in Docker: No

Current Behavior

I use an RSS feed from my indexer to download certain series, for example. If I now select several files for download in the RSS feed in NZBGet, only the top file is downloaded. The other files are marked as "DUPE" and moved to "History". I have to select them manually and select "Download Again" so that they are really downloaded.

The problem occurs with both NZBGet 24.1 and 24.2.

Expected Behavior

NZBGet should download all files right away and not mark all but one as "DUPE".

Steps To Reproduce

No response

Logs

No response

Extra information

Select several files in the RSS feed for download:
Only the first selected entry is downloaded:
The other entries are stored in the history tab as "DUPE":
The other entries must be selected manually in order to actually start them by clicking on "Download Again":

I couldn't able to reproduce the bug, sorry. What extensions are you using? Could they be the cause? Could you send (denis@nzbget.com) the NZBs marked as "DUPE" for testing? They can be found at ${MainDir}/nzb. Thanks.

I do not use any extensions:

NZBGet_extension_manager

I did a few more tests today. At first I thought that the indexer SceneNZBs.com or the the filter 1080p % 2160p for the RSS feed was the cause, as the problem only occurred there with the filter. But then I was able to reproduce the problem with NZBGeek without any filter:

NZBGet_tv_show_test_feed

I have created a GIF where you can see how I manually download several files via the RSS feed, some of which are immediately marked as duplicates:

NZBGet_duplicate_problem

I'm not sure, but apparently the first entries of an episode in the RSS feed are somehow "favored" (in this example, the 1080p version).

The following entries can then be found in the log:

Tue Aug  6 21:41:38 2024    INFO    Collection Dark.Matter.2024.S01E01.Are.You.Happy.in.Your.Life.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested is a duplicate to Dark.Matter.2024.S01E01.Are.You.Happy.in.Your.Life.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024    INFO    Collection Dark.Matter.2024.S01E01.Are.You.Happy.in.Your.Life.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested added to history
Tue Aug  6 21:41:38 2024    INFO    Queue Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024    INFO    Reordering files for Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024    INFO    Queue Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024    INFO    Collection Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested is a duplicate to Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024    INFO    Collection Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested added to history
Tue Aug  6 21:41:38 2024    INFO    Queue Dark.Matter.2024.S01E03.The.Box.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024    INFO    Reordering files for Dark.Matter.2024.S01E03.The.Box.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024    INFO    Queue Dark.Matter.2024.S01E03.The.Box.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024    INFO    Collection Dark.Matter.2024.S01E03.The.Box.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested is a duplicate to Dark.Matter.2024.S01E03.The.Box.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024    INFO    Collection Dark.Matter.2024.S01E03.The.Box.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested added to history
Tue Aug  6 21:41:38 2024    INFO    Queue Dark.Matter.2024.S01E04.The.Corridor.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024    INFO    Reordering files for Dark.Matter.2024.S01E04.The.Corridor.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024    INFO    Queue Dark.Matter.2024.S01E04.The.Corridor.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024    INFO    Collection Dark.Matter.2024.S01E04.The.Corridor.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested is a duplicate to Dark.Matter.2024.S01E04.The.Corridor.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024    INFO    Collection Dark.Matter.2024.S01E04.The.Corridor.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested added to history

Here once again as screenshots:

NZBGet_download_tab

NZBGet_history_tab

The problem does not occur in SABnzbd.

Is this a bug or some function of NZBGet or the indexers?

I need to investigate that more thoroughly. Thank you for the report.

I was able to reproduce such behavior, and I wouldn't say it's a bug. NZBget duplicate detection is quite advanced. In addition, rageid/season/episode it also compares imdb, tvdbid and tvmazeid fields to detect duplicates and in my cases this works well - NZBGet has successfully found the duplicated content and their Duplicate key were identical. To find Duplicate key just click on NZB file name in HISTORY and click on Dupe-> button.

Thanks for the research and the reference to the RSS documentation, from which I also learned some new things. I always thought that duplicate really meant the exact same download/file.

I have also tested it again over the last few days and gained a few new insights. Indeed, the duplicate detection seems to be the cause. If I deactivate the duplicate check under INCOMING NZBS --> DupeCheck, all selected files are always downloaded as expected.

NZBGet always seems to prefer the topmost selected entry in the RSS feed if it is the same episode. In the example from my last post, I downloaded a 1080p version and a 2160p version of the same episode, the duplicate key was then tvdbid=393159-S01-E01:

NZBGet_duplicate_key_second_example

Since the 1080p version is higher up in the RSS feed, it is “preferred” and the 2160p version is marked as a duplicate. So here it works as you described.

With the series from the first post, however, I don't understand why only the imdb value (imdb=28369634) is used as the duplicate key:

NZBGet_duplicate_key_first_example

If you download several (different) episodes, every episode except the top one selected in the RSS feed is then marked as a duplicate. I had a look at the RSS feed of the series, where the tvdbid and the respective season and episode would also be available:

<item>
    <title>Die.Landarztpraxis.S02E69.German.1080p.WEB.h264-WvF</title>
    <guid isPermaLink="true">https://scenenzbs.com/details/b1c0236dce8ed49c5a2f6b2c621aeb2b</guid>
    <link>https://scenenzbs.com/getnzb/b1c0236dce8ed49c5a2f6b2c621aeb2b.nzb&amp;i=5199&amp;r=***MyApiKey***</link>
    <comments>https://scenenzbs.com/details/b1c0236dce8ed49c5a2f6b2c621aeb2b#comments</comments>    
    <pubDate>Sat, 10 Aug 2024 02:03:26 +0200</pubDate>
    <category>TV - DE-HD</category>
    <description><![CDATA[<div><ul><li>ID: b1c0236dce8ed49c5a2f6b2c621aeb2b</li><li>Name: <a href="https://scenenzbs.com/details/b1c0236dce8ed49c5a2f6b2c621aeb2b">Die.Landarztpraxis.S02E69.German.1080p.WEB.h264-WvF</a></li><li>Size: 1.26 GB </li><li>Attributes: Category - <a href="https://scenenzbs.com/browse?t=5140">TV - DE-HD</a></li><li>Groups: <a href="https://scenenzbs.com/browse?g="></a></li><li>Poster: Ds9QuuYtRewZJPiybFmHkPsuSRGW2fLj7i8tv3atWktN</li><li>PostDate: Sat, 10 Aug 2024 02:02:02 +0200</li><li>Password: Unknown</li></ul></div><div style="clear:both;">]]>
    </description>
    <enclosure url="https://scenenzbs.com/getnzb/b1c0236dce8ed49c5a2f6b2c621aeb2b.nzb&amp;i=5199&amp;r=***MyApiKey***" length="1357267947" type="application/x-nzb" />

    <newznab:attr name="category" value="5100" />
    <newznab:attr name="category" value="5140" />
    <newznab:attr name="size" value="1357267947" />
    <newznab:attr name="files" value="14" />
    <newznab:attr name="poster" value="Ds9QuuYtRewZJPiybFmHkPsuSRGW2fLj7i8tv3atWktN" />
    <newznab:attr name="season" value="S02" />
    <newznab:attr name="episode" value="E69" />
    <newznab:attr name="imdb" value="28369634" />
    <newznab:attr name="tvdbid" value="439270" />
    <newznab:attr name="grabs" value="29" />
    <newznab:attr name="comments" value="0" />
    <newznab:attr name="password" value="-1" />
    <newznab:attr name="usenetdate" value="Sat, 10 Aug 2024 02:02:02 +0200" />
    <newznab:attr name="group" value="" />
</item>
<item>
    <title>Die.Landarztpraxis.S02E71.German.1080p.WEB.h264-WvF</title>
    <guid isPermaLink="true">https://scenenzbs.com/details/b2d87184cb7b6618aea2835428115930</guid>
    <link>https://scenenzbs.com/getnzb/b2d87184cb7b6618aea2835428115930.nzb&amp;i=5199&amp;r=***MyApiKey***</link>
    <comments>https://scenenzbs.com/details/b2d87184cb7b6618aea2835428115930#comments</comments>    
    <pubDate>Fri, 09 Aug 2024 23:56:36 +0200</pubDate>
    <category>TV - DE-HD</category>
    <description><![CDATA[<div><ul><li>ID: b2d87184cb7b6618aea2835428115930</li><li>Name: <a href="https://scenenzbs.com/details/b2d87184cb7b6618aea2835428115930">Die.Landarztpraxis.S02E71.German.1080p.WEB.h264-WvF</a></li><li>Size: 1.23 GB </li><li>Attributes: Category - <a href="https://scenenzbs.com/browse?t=5140">TV - DE-HD</a></li><li>Groups: <a href="https://scenenzbs.com/browse?g="></a></li><li>Poster: E75AjdxxQCiJG7gxcTLsx7jA6uLwdPzNcfQ7xoz1TAx1</li><li>PostDate: Fri, 09 Aug 2024 23:53:55 +0200</li><li>Password: Unknown</li></ul></div><div style="clear:both;">]]>
    </description>
    <enclosure url="https://scenenzbs.com/getnzb/b2d87184cb7b6618aea2835428115930.nzb&amp;i=5199&amp;r=***MyApiKey***" length="1317027165" type="application/x-nzb" />

    <newznab:attr name="category" value="5100" />
    <newznab:attr name="category" value="5140" />
    <newznab:attr name="size" value="1317027165" />
    <newznab:attr name="files" value="14" />
    <newznab:attr name="poster" value="E75AjdxxQCiJG7gxcTLsx7jA6uLwdPzNcfQ7xoz1TAx1" />
    <newznab:attr name="season" value="S02" />
    <newznab:attr name="episode" value="E71" />
    <newznab:attr name="imdb" value="28369634" />
    <newznab:attr name="tvdbid" value="439270" />
    <newznab:attr name="grabs" value="33" />
    <newznab:attr name="comments" value="0" />
    <newznab:attr name="password" value="-1" />
    <newznab:attr name="usenetdate" value="Fri, 09 Aug 2024 23:53:55 +0200" />
    <newznab:attr name="group" value="" />
</item>

I don't understand why only the imdb value is taken as the duplicate key here and why this is not supplemented by the season and episode to obtain a unique key. Does the imdb value have priority in NZBGet? Is this perhaps otherwise only used for movies, so that it is not a problem there? I also tried to add the parameter “extended=1” (the indexer uses Newznab), but nothing has changed in the RSS feed, the newznab:attr entries were already there before.

I also took a look at the RSS feed of other series, here usually only the tvdbid value is available (sometimes it is missing), I have not come across the imdb value in any other random samples. This is also the only series so far where I have encountered this problem.

Yes, imdbid is used mostly for movies, and if it exists, it has the highest priority. From the doc:

- for movies: imdb=123456, where 123456 is imdb-id of the title;
- for series: rageid=12345-Sxx-Eyy, where 12345 is rageid of the series, xx is season number and yy is episode number. If rageid isn’t available, fields tvdbid and tvmazeid can be used instead.

But in your case, the series lists both imdb and tvdbid, which seems weird to me. To be sure, you checked that the downloaded content really isn't the same, right? If so, then I'll think about how to improve the Dupe detection algorithm.

extended=1 doesn't work for me either. Never used it before.

nzbgetcom / nzbget