Mirabis Fork: OMGWTFNZBS & Gingadaddy

[albino1]

I really appreciate Mirabis taking up the reigns to get this thing up to speed. I wasn't sure if this Issues page is the correct one to put questions about his version, so if it's not, point me in the right direction and I'll move over there.

Most of the "Newznab Host" API-based searches seem to work fine (though some indexers have their own problems unrelated to NZBMegaSearch) however, under "Other NZB engines" OMG fails about half the time and Gingadaddy doesn't work at all.

The error from OMG is:

ERROR (100): Incorrect user credentials (Omgwtfnzbs)

And in the log it says:

SearchModule - INFO - Wrong api/pass https://omgwtfnzbs.org/ 1.68499994278

The info is definitely correct as often times it does return results. This is probably an OMG issue though, and not related to NZBMegaSearch, but I thought I'd mention it just in case there's something else going on since it's got its own special field.

The Gigadaddy errors with an SSL problem though:

ERROR (400): Generic server error (gingadaddy.com)

And from the log:

DeepsearchModule - WARNING - Fetched exception: https://www.gingadaddy.com - <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>

Which makes me think that one is broken in some sort of similar way that maybe the NZBClub one was broken before it was updated. Or maybe their server is just not setup properly right now as their page says they recently had a data crash.

Anyway, I think this program is really great an underrated, so I'm glad to see it being developed again.

[Mirabis]

I'll have to run some debugging and check why those are no longer working. I don't even know why he used a custom approach for them (except for gingadaddy.com) at the moment.

You could use their API though :)

[albino1]

Thanks for replying!

I tried setting up GingaDaddy as an API, but the system is kind of strange, which is why I suspect he went a different route.

When you first sign up you're not given an API key. You have to create an RSS feed and choose the categories you'd like included, after which an API key is generated for you. However, even using that API key, NZBMegaSearch returns no results no matter what you search for. Even things that are explicitly in the RSS feed aren't found via the API. All it says is:

SUCCESS (200): gingadaddy.com in 0.5s, no results 

The URL I'm using is:

https://www.gingadaddy.com

And I've tried it both with and without SSL.

Maybe I'm doing something wrong on my end though (incorrect URL?), but there's only a few options so I don't know what I could be missing.

[albino1]

I should add that using OMG via the Newznab API option doesn't work either. Even though it's saved and marked active the response is:

0 total (0 failed, 0 successful)

Maybe because it's not built on the Newznab platform.

[albino1]

Just a small update: apparently some indexers block NZBMegaSearch (no idea why) and so I tried modifying the User Agent string to see if that would work, but it didn't change anything. The files I modified were:

ApiModule.py
DeepsearchModule.py
miscdefs.py
SuggestionModule.py

But maybe I missed one, or didn't do it right.

[albino1]

Also, I've noticed that some indexers don't work at all (i.e. PirateNZB), and others are missing some results (i.e. NZBPlanet). I think the problem is likely on their ends, but just in case they had some sort of block in place I tested them with the modified user agent as well, but no luck.

[tifoji]

Wow first time I am hearing that NZBM is blocked by some indexers. Pretty sure the Top 5 do not block it as I always get results from them. If you suspect someone is blocking, you can ask them nicely in IRC. Doesn't make sense still, even those site owners probably use it :+1:

Is PirateNZB new? Never heard of it before and I have heard and am member of most of the infamous ones I'd say.

[albino1]

PirateNZB is fairly new, maybe like 6 months old. I wouldn't really recommend it, but I wanted to add some easy nZEDb based indexers just in case that helped, and somebody had previously sent me an invite to it.

I can only assume that they're blocking NZBMegaSearch specifically as they return no results, otherwise it would be worthless for Sickbeard/Sonarr/etc as well and people would be complaining.

Certainly some sites do explicitly block NZBMegaSearch. Supposedly nzbs.in does, but I'm not a member so I can't confirm. And here's a thread on Reddit talking about nzb.su blocking it:

http://www.reddit.com/r/usenet/comments/2sh7by/nzbmegasearch_nzbsu/

[tifoji]

nzb.su I didn't renew their membership so it must have slipped my results. It used to work but maybe they have blocked it now.

nzbs.in doesn't block NZBM. They just banned SickRage starting today and that's about it.

[albino1]

Seems like a slippery slope. If you're going to block SickRage, then why not SickBeard and Sonarr? And why not NZBM which you can use with those programs to theoretically bypass the block? Hopefully it's all just misinformation though, and nobody is truly blocking anybody. Maybe the sites that don't work with NZBM are just outright broken :)

[tifoji]

Yah true. But the SickRage ban on nzbs.in is real.

"Furthermore, SickRage doesn't seem to have any throttling mechanism like other apps do.According to one of SickRage's developers,"SR is so popular that we were hogging so many resources that their own app could not get results reliably." Therefore, we recommend that people using SickRage switch to Sonarr. And whether you take our recommendation or not, we don't care. The moral of the story is SickRage will no longer work on NZBs.in."

In almost all cases, it is just a bad apple or two causing the indexers to take such action, not necessarily the app. NZBM is so niche that they have not had any reason to implement bans on it. Sites that don't work probably is an issue both on the index and on NZBM, which is why we need active development :)

[albino1]

Sites that don't work probably is an issue both on the index and on NZBM, which is why we need active development :)

Couldn't agree more!

Now if only I was capable of contributing something more than "wouldn't this be a cool feature" and "this is broke, somebody fix it" :)

[Mirabis]

So

• PirateNZB
• Nzbplanet*
• nzb.su*
• nzbs.in

need another approach if I understand correctly? Not sure if someone already went ahead and did some debugging, but I'll be able to start working on it after 6 July.

• Don't have VIP/API access on those at the moment.

[fatguy666]

@albino1 Just wanted to point out, I have gingadaddy working fine with my nzbmegasearch and working through sickrage.

In DeepsearchModule.py change the url from https to http for gingadaddy.

User credentials go into the gingadaddy box in megasearch.

[donkeyhigh]

Could someone PLEASE make it so there's only 1 search button, which searches all added sites. Or a option to tick off which sites should be searched with normal search, and which would be searched with extended search.

I only use extended, so making that primary would be nice, so I can just hit "enter" after writing my search. On Jun 24, 2015 19:48, "fatguy666" notifications@github.com wrote:

Just wanted to point out, I have gingadaddy working fine with my nzbmegasearch and working through sickrage.

In DeepsearchModule.py change the url from https to http for gingadaddy.

User credentials go into the gingadaddy box in megasearch.

— Reply to this email directly or view it on GitHub https://github.com/pillone/usntssearch/issues/152#issuecomment-114953886 .

[albino1]

@fatguy666

I just tried that, and it didn't work for me. Are there any additional lines in other modules? All I found was line 464.

I changed it to http, and restarted NZBM, but it still gives the same error.

Edited: I take it all back, it does work now. I'm not sure what I did wrong the first time.

So: I can confirm fatguy666's 'fix' of disabling SSL and just using http works if Mirabis wants to add that officially. Though it's annoying that SSL doesn't work with their site.

 

@donkeyhigh

Under the Timeout drop down if you leave it to Normal then it's the first button, and if you change it to Slow Response it's the second button. If you want to change the amount of time actually being used then in builtin_params.ini you can modify these lines:

default_timeout = 12
timeout_slow = 20

If you think about it, you could just change default_timeout to 20 and then that would effectively act like Extended Search, while still allowing you to use the ENTER button. You could then set any you don't always want to use to Slow Response, and they would only get used when you use the second button.

That's effectively how I use it right now. I set the indexes I always want to query to Normal so that they get used when I hint ENTER and then I set indexes with things like limited API calls to Slow Response so they only get queried if I manually hit the second button.

FWIW, I've changed my Slow Response to 30 seconds because some APIs (like NZBGeek) often timeout otherwise.

[albino1]

@Mirabis

I can't confirm nzbs.in as I don't have an account either, that's just from reading stuff on Reddit, so who knows what's really true.

I can say for certain that the NZBPlanet API is missing results compared to the normal index search, though why that is I have no idea. And I can say that for me, the PirateNZB API returns zero results no matter what I search for. It always just says:

SUCCESS (200): piratenzb.com in 1.3s, no results 

That problem might be on their end though.

I'd be more than happy to send you an invite to NZBPlanet and provide you with some examples if you want to try and debug it. Just let me know.

[albino1]

Also, I should add that since my original conversation with siuside about nzb.su I got an account and can confirm that it works just fine using NZBM, so there's nothing to worry about there. That Reddit thread I linked to is 5 months old, so if it was ever blocked then maybe they've unblocked it since then.

[fatguy666]

I have nothing personally to do with gingadaddy but after the last problem with the server they abandoned https as it was so slow, although they still use it for subscribing.

The slowness was part of the reason that gingadaddy only worked with extended search but since they stopped using it the search times are reasonable.

[albino1]

That explains a lot. Thanks for the info!

[donkeyhigh]

Wait what? I can't get gingadaddy to work.. Do I have to update my megasearch or something? Because it won't update correctly. I use win7x64..

Sorry for the newbishness.. On Jun 25, 2015 11:32 AM, "fatguy666" notifications@github.com wrote:

I have nothing personally to do with gingadaddy but after the last problem with the server they abandoned https as it was so slow, although they still use it for subscribing.

The slowness was part of the reason that gingadaddy only worked with extended search but since they stopped using it the search times are reasonable.

— Reply to this email directly or view it on GitHub https://github.com/pillone/usntssearch/issues/152#issuecomment-115185275 .

[Mirabis]

@donkeyhigh You'll have to edit DeepsearchModule.py line 464 to turn https:// into http://

Alternative is to wait for me to push a new update, but there are some additional fixes I want to include before uploading.

[Mirabis]

Have been debugging PirateNZB all day, but think I'll have to contact them.. 04:00 AM and still not working... once u specify &q={query} into the api it returns [] as data. Maybe I should retrieve full output and filter as workaround..

[albino1]

I would imagine that Sickbeard/Sonarr/etc use the exact same methodology, so does that mean that there's no results for those programs as well? If so, then their site is just broken.