evilhero / mylar

An automated Comic Book downloader (cbr/cbz) for use with SABnzbd, NZBGet and torrents
GNU General Public License v3.0
976 stars 170 forks source link

Possible bug in searching newznab #1746

Closed miko3ed closed 3 years ago

miko3ed commented 7 years ago

Getting some strange searching issues with newznab provider NZBGeek. I added X-Force (2010) and it isn't finding any issues from the indexer api. Here is an example of the URL mylar is using (from the debug logs):

https://api.nzbgeek.info/api?t=search&q=x-force%20025&o=xml&cat=7030&apikey=xUDONTNEEDTOKNOWTHISx

Nothing appears wrong in the url, but it returns no results. If you change the - in x-force to %20 there are results from the API.

Is this an issue with NZBGeek?

evilhero commented 7 years ago

It shouldn't be an issue with either geek or mylar as far as I'm aware. As long as the issue was posted to the groups in the same format (ie. 'X-force' and not 'X force').

I'm nowhere near my machine atm and won't be for a few hours so I can't even test - what is the filename /header on geek say when you do a search via the website?

miko3ed commented 7 years ago

If I do a search for "x-force 25" via the website this is the URL that gets created:

https://nzbgeek.info/geekseek.php?moviesgeekseek=1&browsecategory=7030&browseincludewords=x-force+25

If I do a search for "x force 25" this is the url:

https://nzbgeek.info/geekseek.php?moviesgeekseek=1&browsecategory=7030&browseincludewords=x+force+25

In both cases the results returned on the page are the same records.

This is what the results from the API look like for the two searches:

URL: https://api.nzbgeek.info/api?t=search&q=x-force%20025&o=xml&cat=7030&apikey=xUDONTNEEDTOKNOWTHISx

<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:newznab="http://www.newznab.com/DTD/2010/feeds/attributes/" version="2.0">
<channel>
<newznab:response offset="0" total="0"/>
</channel>
</rss>

Of note, this produces the same results as the results via manual search through the webpage.

URL: https://api.nzbgeek.info/api?t=search&q=x%20force%20025&o=xml&cat=7030&apikey=xUDONTNEEDTOKNOWTHISx

<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:newznab="http://www.newznab.com/DTD/2010/feeds/attributes/" version="2.0">
<channel>
<atom:link href="https://api.nzbgeek.info/api?t=search&q=x%20force%20025&o=xml&cat=7030&apikey=e72facc8cc555acdd392270a59dc65bd" rel="self" type="application/rss+xml"/>
<title>api.nzbgeek.info</title>
<description>NZBgeek API</description>
<link>http://api.nzbgeek.info/</link>
<language>en-gb</language>
<webMaster>info@nzbgeek.info (NZBgeek)</webMaster>
<category/>
<image>
<url>https://api.nzbgeek.info/covers/nzbgeek.png</url>
<title>api.nzbgeek.info</title>
<link>http://api.nzbgeek.info/</link>
<description>NZBgeek</description>
</image>
<newznab:response offset="0" total="5"/>
<item>
<title>X-Force 025 (2010) (digital) (Kingpin-Empire)</title>
<guid isPermaLink="true">
https://api.nzbgeek.info/details/0fc4619b7ce8c288c968649c982668d7
</guid>
<link>
https://api.nzbgeek.info/api?t=get&id=0fc4619b7ce8c288c968649c982668d7&apikey=e72facc8cc555acdd392270a59dc65bd
</link>
<comments>
https://nzbgeek.info/geekseek.php?guid=0fc4619b7ce8c288c968649c982668d7
</comments>
<pubDate>Tue, 25 Mar 2014 04:48:43 +0000</pubDate>
<category>Books > Comics</category>
<description>X-Force 025 (2010) (digital) (Kingpin-Empire)</description>
<enclosure url="https://api.nzbgeek.info/api?t=get&id=0fc4619b7ce8c288c968649c982668d7&apikey=e72facc8cc555acdd392270a59dc65bd" length="48842000" type="application/x-nzb"/>
<newznab:attr name="category" value="7000"/>
<newznab:attr name="category" value="7030"/>
<newznab:attr name="size" value="48842000"/>
<newznab:attr name="guid" value="0fc4619b7ce8c288c968649c982668d7"/>
</item>
<item>
<title>X-Force 025 (1993) (Digital) (Meganubis-Empire)</title>
<guid isPermaLink="true">
https://api.nzbgeek.info/details/dbbec1d98b59d28df187e89237916d0a
</guid>
<link>
https://api.nzbgeek.info/api?t=get&id=dbbec1d98b59d28df187e89237916d0a&apikey=e72facc8cc555acdd392270a59dc65bd
</link>
<comments>
https://nzbgeek.info/geekseek.php?guid=dbbec1d98b59d28df187e89237916d0a
</comments>
<pubDate>Mon, 16 Dec 2013 01:22:13 +0000</pubDate>
<category>Books > Comics</category>
<description>X-Force 025 (1993) (Digital) (Meganubis-Empire)</description>
<enclosure url="https://api.nzbgeek.info/api?t=get&id=dbbec1d98b59d28df187e89237916d0a&apikey=e72facc8cc555acdd392270a59dc65bd" length="91132000" type="application/x-nzb"/>
<newznab:attr name="category" value="7000"/>
<newznab:attr name="category" value="7030"/>
<newznab:attr name="size" value="91132000"/>
<newznab:attr name="guid" value="dbbec1d98b59d28df187e89237916d0a"/>
</item>
<item>
<title>Uncanny X-Force 025 (2012) (Digital) (Zone-Empire)</title>
<guid isPermaLink="true">
https://api.nzbgeek.info/details/339e4e82fa5ca50643a8918f73bbd43a
</guid>
<link>
https://api.nzbgeek.info/api?t=get&id=339e4e82fa5ca50643a8918f73bbd43a&apikey=e72facc8cc555acdd392270a59dc65bd
</link>
<comments>
https://nzbgeek.info/geekseek.php?guid=339e4e82fa5ca50643a8918f73bbd43a
</comments>
<pubDate>Wed, 30 Jan 2013 20:08:03 +0000</pubDate>
<category>Books > Comics</category>
<description>Uncanny X-Force 025 (2012) (Digital) (Zone-Empire)</description>
<enclosure url="https://api.nzbgeek.info/api?t=get&id=339e4e82fa5ca50643a8918f73bbd43a&apikey=e72facc8cc555acdd392270a59dc65bd" length="59118000" type="application/x-nzb"/>
<newznab:attr name="category" value="7000"/>
<newznab:attr name="category" value="7030"/>
<newznab:attr name="size" value="59118000"/>
<newznab:attr name="guid" value="339e4e82fa5ca50643a8918f73bbd43a"/>
</item>
<item>
<title>cw-superheroes [037170] X-Force 025</title>
<guid isPermaLink="true">
https://api.nzbgeek.info/details/cabef0652f962e950d03f26078f9fcd2
</guid>
<link>
https://api.nzbgeek.info/api?t=get&id=cabef0652f962e950d03f26078f9fcd2&apikey=e72facc8cc555acdd392270a59dc65bd
</link>
<comments>
https://nzbgeek.info/geekseek.php?guid=cabef0652f962e950d03f26078f9fcd2
</comments>
<pubDate>Sun, 17 Mar 2013 12:06:36 +0000</pubDate>
<category>Books > Comics</category>
<description>cw-superheroes [037170] X-Force 025</description>
<enclosure url="https://api.nzbgeek.info/api?t=get&id=cabef0652f962e950d03f26078f9fcd2&apikey=e72facc8cc555acdd392270a59dc65bd" length="7432000" type="application/x-nzb"/>
<newznab:attr name="category" value="7000"/>
<newznab:attr name="category" value="7030"/>
<newznab:attr name="size" value="7432000"/>
<newznab:attr name="guid" value="cabef0652f962e950d03f26078f9fcd2"/>
</item>
<item>
<title>cw-superheroes [025170] X-Force 013</title>
<guid isPermaLink="true">
https://api.nzbgeek.info/details/3e4df7e6d9e6cb081021795859d43ea4
</guid>
<link>
https://api.nzbgeek.info/api?t=get&id=3e4df7e6d9e6cb081021795859d43ea4&apikey=e72facc8cc555acdd392270a59dc65bd
</link>
<comments>
https://nzbgeek.info/geekseek.php?guid=3e4df7e6d9e6cb081021795859d43ea4
</comments>
<pubDate>Sun, 17 Mar 2013 12:06:36 +0000</pubDate>
<category>Books > Comics</category>
<description>cw-superheroes [025170] X-Force 013</description>
<enclosure url="https://api.nzbgeek.info/api?t=get&id=3e4df7e6d9e6cb081021795859d43ea4&apikey=e72facc8cc555acdd392270a59dc65bd" length="4404000" type="application/x-nzb"/>
<newznab:attr name="category" value="7000"/>
<newznab:attr name="category" value="7030"/>
<newznab:attr name="size" value="4404000"/>
<newznab:attr name="guid" value="3e4df7e6d9e6cb081021795859d43ea4"/>
</item>
</channel>
</rss>
evilhero commented 7 years ago

It looks like the search problem is specific to nzbgeek - not exactly sure why atm tho. I tested the search query against a few different providers and they all returned search results that resulted in mylar snatching the nzb (all newznab-based).

I'll do some more testing against geek - it might require a special add to a query just for that particular indexer... Or the query might have to be urlencoded prior (it doesn't do that atm I believe)

miko3ed commented 7 years ago

Yeah, I kinda assumed it would be with them. I could be wrong but I think you are URLEncoding, I took a quick glance at the search code, but also hyphens are URL valid so don't get encoded normally. I also did a search attempt through curl changing the - to a %2D and got no results

miko3ed commented 7 years ago

Doing some more testing, it definitely looks like geeks api doesn't like hyphens. I think the web gui is converting hyphens to spaces but the API isn't. I just tested spider-gwen and am seeing the same results as above with x-force. If I specify "spider gwen" as an alternate search in mylar it is picking up correct issues from the geek api. Just FYI :)

miko3ed commented 7 years ago

I posted on the geek support forums about this behavior, will let you know if I get any productive responses

miko3ed commented 7 years ago

Looks like this is a known issue with geek, not sure if they will fix it. I've switched over to usenet-crawler.com for my comic searches and that seems to be working well for now

miko3ed commented 7 years ago

As an aside, is there any way I can easily see what comics I've setup Alternate Search Names for?

evilhero commented 7 years ago

Great work on the detective stuff - saves me a lot of time figuring things out on this end ;)

If it's an outstanding problem at geek, then I might just add in something so that the searches will return proper results since I don't think it would be that difficult to implement.

There is no way to see all your alternate search names in a concise manner - the only way is to go into each series and visually check.

However, I do like the notion of being able to manage/view all the alternate search names (ie. Aliases) in a bulk type of manner - so this is something I'd try and add going forward, once in an dedicate some time to coding that. If you wanted, you could add an enhancement request issue for this just so I can track things better (I tend to forget if I can't see the item in the issues list).

miko3ed commented 7 years ago

Happy to do detective work like this :)

I put in an enhancement request.