evilhero / mylar

An automated Comic Book downloader (cbr/cbz) for use with SABnzbd, NZBGet and torrents
GNU General Public License v3.0
976 stars 172 forks source link

https://api.nzbgeek.info/api?t=get&id Blah blah downloads SOME stuck in Sab #1011

Closed stustaff closed 3 years ago

stustaff commented 9 years ago

So ant man 1 came through ok and ant man 5 not so much just sat in sab saying https://api.nzbgeek.info/api?t=get&id= Removed &apikey= removed &i=&r=

i have maybe 20 just sat and 5 that downloaded as normal..?

i updated to python 2.7.9 as per earlier ssl issue seems like SAB dosnt play nicely with that so am I stuck? 2.7.8 mylar doesnt work, 2.7.9 SAb doesnt work.

ALthough all fine with sickbeard?!

changing all to http now works add https and breaks down again. It says all worked in mylar but in SAB its creating an error starting with "Empty NZB file api?" then same info as above.

stustaff commented 9 years ago

Just looked in log and noticed this when i hit retry... coincidence?

Uncaught exception: Traceback (most recent call last): File "/Applications/mylar-master/mylar/logger.py", line 159, in new_run old_run(_args, *_kwargs) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run self.*target(_self.__args, _self.__kwargs) File "/Applications/mylar-master/mylar/webserve.py", line 1165, in retryissue sendit = search.searcher(Provider, nzbname, comicinfo, link=link, IssueID=IssueID, ComicID=ComicID, tmpprov=fullprov, directsend=True) File "/Applications/mylar-master/mylar/search.py", line 1652, in searcher name_newznab = newznab[0].rstrip() TypeError: 'NoneType' object has no attribute '__getitem**'

evilhero commented 9 years ago

Can you post the complete link that the send-to-sab is generating (just xxxx out your api keys) for one of the download that isn't working? The link it's generating probably isn't correct (or keys are duplicated which wouldn't work either).

Don't use retry either.. It tries to resend the last successful send which is messed up at this point. Something I'll be fixing, but in the short term just avoid using it.

stustaff commented 9 years ago

is this what you mean? seemd to have API in loads? first is when it returns an Empty result in SAb

Empty NZB file api?t=get&id=xxxxxxNotAPIxxxxxapikey=xxthiswasapixxx&i=&r=xxthiswaspixx.nzb [https://api.nzbgeek.info/api?t=get&id=xxxnotapixxxapikey=xxxthiswasapixxx&i=&r=xxxthiswasapixxx]

This is what the original issue had in sab that kept retrying Trying to fetch NZB from https://api.nzbgeek.info/api?t=get&id=9cc56deletedb42631f56badf167be&apikey=30ee748ddeleted2de740e2f9dc686b&i=&r=30ee748d38e2deleted0e2f9dc686b6

evilhero commented 9 years ago

Close, in the mylar.log file it will actually be called send-to-sab... I need to see that complete url.

stustaff commented 9 years ago

mylar.log finds nothing if i search even for just sab? no entry with send-to-sab but im assuming log is short as I closed and reopened Mylar this was in the GUI log sab-to-send:http://localhost:8083/api?apikey=cbb2f78removed21919d90a9&mode=addurl&name=https%3A%2F%2Fapi.nzbgeek.info%2Fapi%3Ft%3Dget%26id%3D644cfe8removed3b9ed933bc1f%26apikey%3D30ee74removed740e2f9dc686b%26i%3D%26r%3D30ee7removedf9dc686b6&priority=-100&cat=comics&script=ComicRN.py

stustaff commented 9 years ago

I was able to create this full from mylar.log

10-May-2015 23:13:56 - DEBUG :: Thread-12 : Completed send-to-SAB link: https://localhost:8083/api?apikey=xxxxxx&mode=addurl&name=https%3A%2F%2Fapi.nzbgeek.info%2Fapi%3Ft%3Dget%26id%3Df4xxxxxxx08dce1d29dab%26apikey%3Dxxxxxx686b%26i%3D%26r%3D30eexxxxx740e2f9dc686b6&priority=-100&cat=comics&script=ComicRN.py 10-May-2015 23:13:56 - INFO :: Thread-12 : sab-to-send:https://localhost:8083/api?apikey=xxxxx19d90a9&mode=addurl&name=https%3A%2F%2Fapi.nzbgeek.info%2Fapi%3Ft%3Dget%26id%3Df4xxxxxxxx1c208dce1d29dab%26apikey%3Dxxxxxxx9dc686b%26i%3D%26r%3D30eexxxxx2f9dc686b6&priority=-100&cat=comics&script=ComicRN.py 10-May-2015 23:13:56 - ERROR :: Thread-12 : Unable to send nzb file to SABnzbd

stustaff commented 9 years ago

if i paste that into browser i get ok in window and SAB tells me its added to queue, but then gives a warning of empty NZB file.

evilhero commented 9 years ago

It's putting the apikey in twice.. So it's an invalid link. I'll try and get a fix up abit later as I can't get to a machine until then.

Can you confirm what versions of python and sabnzbd you're running?

stustaff commented 9 years ago

yeh python 2.7.9 osx Sab 0.7.20

thanks

evilhero commented 9 years ago

Ok that's good.. I can't verify the fix works 100% as I don't use nzbgeek, but I'm pretty sure it'll not republication the api keys. Fix will be up in development very shortly

stustaff commented 9 years ago

Thanks for taking time to review. Appreciated and donation sent towards a couple of drinks to help deal with fools like me. :)

evilhero commented 9 years ago

That's great - thanks for the support and contributing to my addiction to Java ;)

If the fix works for you (or not) , please let me know either way so if something needs to be addressed it can be looked at, or otherwise we can close the issue of it's all worky-worky.

stustaff commented 9 years ago

No problem I'm away for a few days but will let you know and close this off if it resolves.

stustaff commented 9 years ago

I updated Mylar and am seeing same issue.

13-May-2015 09:35:12 - DEBUG :: Thread-13 : sab-to-send:http://localhost:8083/api?apikey=cbbdeleted&mode=addurl&name=https%3A%2F%2Fapi.nzbgeek.info%2Fapi%3Ft%3Dget%26id%3D2deleted28ee279e9c2ee502c6c0%26apikey%3D30ee748ddeletede2f9dc686b&priority=-100&cat=comics&script=ComicRN.py 13-May-2015 09:35:13 - INFO :: Thread-13 : Successfully sent nzb file to SABnzbd

evilhero commented 9 years ago

And sabnzbd isn't receiving any nzbs or they're empty?

stustaff commented 9 years ago

They are empty

On 13 May 2015, at 18:47, evilhero notifications@github.com wrote:

And sabnzbd isn't receiving any nzbsu or they're empty?

— Reply to this email directly or view it on GitHub.

evilhero commented 9 years ago

If you copy/paste the addurl&name link in a very, does it add it ok to your sabnzbd?

Also is it an rss hit that's it's grabbing or via the api of nzbgeek?

stustaff commented 9 years ago

I will test, how do I check the second thing?

On 13 May 2015, at 19:27, evilhero notifications@github.com wrote:

If you copy/paste the addurl&name link in a very, does it add it ok to your sabnzbd?

Also is it an rss hit that's it's grabbing or via the api of nzbgeek?

— Reply to this email directly or view it on GitHub.

evilhero commented 9 years ago

In the log just prior to that link that you pasted it would say somewhere that's it's from RSS or nothing if it's am api hit (the shhh line also indicates which)

stustaff commented 9 years ago

API not RSS and pasting - https://api.nzbgeek.info/api?t=get&id=6dcb1b3190f02a883e8a3df8dad3b24c&apikey=30ee7delete0e2f9dc686b6 into browseradds to Sab fine

evilhero commented 9 years ago

What about the full send to sab link from your browser?

It's working on my end, or was yesterday, so I'm really abit confused as to why it's not working atm.

stustaff commented 9 years ago

me too. I just retried the one from yesterday and it went through and was added instantly!

I havent reset anything or changed anything at all? ill let it run for a couple of days and then report back.

stustaff commented 9 years ago

Oh dear... then tried another and it didnt I can replicate with these two - issue 16 no(it has been downloaded previously) issue 14 yes!

this was from log for issue 16 14-May-2015 14:00:54 - DEBUG :: Thread-23 : Sending request to RSS for Wolverines : 016 (2015) 14-May-2015 14:00:54 - DEBUG :: Thread-23 : send-to-SAB host &api initiation string : http://localhost:8083/api?apikey=cbb2f7delete252421919d90a9 14-May-2015 14:00:54 - DEBUG :: Thread-23 : Completed send-to-SAB link: http://localhost:8083/api?apikey=cbb2f78delete21919d90a9&mode=addurl&name=https%3A%2F%2Fapi.nzbgeek.info%2Fapi%3Ft%3Dget%26id%3D8b43ddelete77b500ecba20%26apikey%3D30ee74delete9dc686b&priority=-100&cat=comics&script=ComicRN.py 14-May-2015 14:00:54 - DEBUG :: Thread-23 : sab-to-send:http://localhost:8083/api?apikey=cbb2f7delete1919d90a9&mode=addurl&name=https%3A%2F%2Fapi.nzbgeek.info%2Fapi%3Ft%3Dget%26id%3D8b43delete77b500ecba20%26apikey%3D30ee7deletef9dc686b&priority=-100&cat=comics&script=ComicRN.py

i also pasted into browser and it trys to send to Sab but sab reports same thing if generated by mylar or link into browser

2015-05-14 13:51:40,252 WARNING: Empty NZB file api?t=get&id=8b43db8929a013b4703077b500ecba20&apikey=30eedelete740e2f9dc686b.nzb [https://api.nzbgeek.info/api?t=get&id=8b43dbdelete077b500ecba20&apikey=30ee74delete40e2f9dc686b] 2015-05-14 13:45:34,701 WARNING: Empty NZB file api?t=get&id=8b43db8929a013b4703077b500ecba20&apikey=30ee7deletede740e2f9dc686b.nzb [https://api.nzbgeek.info/api?t=get&id=8b43dbdeleteb4703077b500ecba20&apikey=30ee7delete40e2f9dc686b]

here is log for issue 14

14-May-2015 13:58:49 - DEBUG :: Thread-22 : Sending request to RSS for Wolverines : 014 (2015) 14-May-2015 13:58:50 - DEBUG :: Thread-22 : send-to-SAB host &api initiation string : http://localhost:8083/api?apikey=cbb2deleteb91252421919d90a9 14-May-2015 13:58:50 - DEBUG :: Thread-22 : Completed send-to-SAB link: http://localhost:8083/api?apikey=cbb2f785delete252421919d90a9&mode=addurl&name=https%3A%2F%2Fapi.nzbgeek.info%2Fapi%3Ft%3Dget%26id%3D6dcb1b31delete8a3df8dad3b24c%26apikey%3D30ee7delete740e2f9dc686b6&priority=-100&cat=comics&script=ComicRN.py 14-May-2015 13:58:50 - DEBUG :: Thread-22 : sab-to-send:http://localhost:8083/api?apikey=cbb2f7delete252421919d90a9&mode=addurl&name=https%3A%2F%2Fapi.nzbgeek.info%2Fapi%3Ft%3Dget%26id%3D6dcb1delete8a3df8dad3b24c%26apikey%3D30ee748delete0e2f9dc686b6&priority=-100&cat=comics&script=ComicRN.py

evilhero commented 9 years ago

Is it pulling both of those from the rss or from the api? If ones working vs not, then I would suspect one is rss and the other is api, and one of them is causing the problem.

If they're both from rss, then it's possible that the one (issue 16) has an invalid link - I remember something about nzbgeek leaving the nzbs intact but it's marked as spam.

Can you verify via their site if it's a valid nzb still?

stustaff commented 9 years ago

still valid on site. I have done a sample and a few worked fine rest not. all worked fine according to Mylar. in SAB three worked and have been added to queue and 20 have appeared a sa warning saying empty.

I cant se any common reaon why.?

remember i hadnt had any issues until i upgraded python.

is it worth a total reinstall? anyway to keep a record of my wanted list and stuff?

evilhero commented 9 years ago

Well python 2.7.9 is required for sites that use https and have SNI (Server Name Indication) enabled and are on a shared host as it needs to verify the certificate against the right site. Downgrading to a different versions of python would just cause different problems (aside from OS ones since python is linked directly into the OS for nix based systems).

It might be due to rss vs api results being downloaded, it's just trying to figure out which. I'm on python 2.7.9 and ubuntu 12.04 and it's just required almost if you hit https sites due to the SNI and every site using cloudflare type services (they all use SNI).

You can try doing this. Within mylar, turn off rss feeds in the configuration/search providers tab. Save it and restart Mylar. Then try doing a manual search (or two) for something and see if it passes it (turning off rss will force it to use only api results).

If you want to retain your existing stuff (for backups), you need to backup your mylar.db, config.ini and your entire cache directory (where mylar stores all the comic covers).

stustaff commented 9 years ago

ok so first 7 i have tried with RSS off all added to sab fine??

evilhero commented 9 years ago

Ok so then it's a problem with the rss caching most likely. Is it occurring across all your providers? I only recall you mentioning nzbgeek, have you tried any others (experimental)?

DGMayor commented 9 years ago

I just noticed I'm having issues with NZBGeek as well. When I copy the URL in the error message from SABnzbd, it's apparently truncating the last character of my API key. I verified that it was entered properly in the settings (hasn't changed).

evilhero commented 9 years ago

Which one is getting cut off - the apikey for sab or the apikey for nzbget?

DGMayor commented 9 years ago

The Error message in Sab of empty nzb file shows the link with the last character of my nzbgeek api cut off.

evilhero commented 9 years ago

What about within the mylar.log file - does it have the complete link in the send to sab portion?

DGMayor commented 9 years ago

Scrolled away already via web interface (not at my computer), but subsequent attempts show the complete key there. (I've got another issue with NZB.SU I'm about to post). FYI all these issues started up this week. Things were fine a week or so ago if that means anything to you.

evilhero commented 9 years ago

Yeah I had to switch mylar to the requests library due to problems with SNI support.. So some of these problems are expected if you're trying to access an https site and you're on python 2.7.6 or lower (since SNI support is only available in >2.7.7).

Newest development commit though has the updated requests library which might help things.

DGMayor commented 9 years ago

I'm on 2.7.9 and have been for a while and I'm on the latest dev commit.

evilhero commented 9 years ago

So you shouldn't be having any connection issues since 2.7.9 is able to use requests.

Your issue is probably the link itself, or are you using https with sabnzbd?

DGMayor commented 9 years ago

No I am not.

evilhero commented 9 years ago

Can you paste in the fill url that it's sending to your sab? The complete line with your sabnzbd host in it is the one I need to see (just xxxx out your keys)

DGMayor commented 9 years ago

Is there another log location other than the logs folder within the mylar folder? There are two logs in there both dated last friday...

evilhero commented 9 years ago

Nope that'd be the only place unless you changed the log location, or it errored out and hasn't been logging due to the error.

DGMayor commented 9 years ago

Mylar.log contains entries from 4:53AM to 7:17AM on May 15th Mylog.log.5 contains entries from 2:12AM to 4:53am on May 15th

That's my current log folder.

evilhero commented 9 years ago

I'd restart mylar and /or shut it down and then start it up and see if it starts logging without errors. Something errored out and caused it to stop logging.

DGMayor commented 9 years ago

Is there a way to unmark it as failed? It won't try the nzbgeek file anymore because I had marked it as failed before I realized there might be a bigger issue.

evilhero commented 9 years ago

Unfortunately not without editing the sql manually. It was in the pipe, but then the requests became more prevalent so it took precedence in the hierarchy.

Will have something for this in the next day or so as I find myself having this problem as well occasionally.

DGMayor commented 9 years ago

So I restarted and now it won't load anymore. https://github.com/evilhero/mylar/issues/1020

evilhero commented 9 years ago

This bug should be squashed in the next development commit - hopefully early tomorrow.

DGMayor commented 9 years ago

Still having issues with nzbgeek.

Uncaught exception: Traceback (most recent call last):

File "C:\Mylar\mylar\logger.py", line 159, in new_run old_run(_args, *_kwargs) File "C:\Python27\lib\threading.py", line 763, in run self.*target(_self.__args, _self.__kwargs) File "C:\Mylar\mylar\webserve.py", line 1165, in retryissue sendit = search.searcher(Provider, nzbname, comicinfo, link=link, IssueID=IssueID, ComicID=ComicID, tmpprov=fullprov, directsend=True) File "C:\Mylar\mylar\search.py", line 1684, in searcher name_newznab = newznab[0].rstrip() TypeError: 'NoneType' object has no attribute '__getitem**' 2015-05-21 12:59:02 INFO link : https://01100111011001010110010101101011.info/getnzb/64408d0b883a7b21cc1951be62675161.nzb&i=7030&r=myapikey 2015-05-21 12:59:02 INFO newznab detected as : NZBGeeks @ https://01100111011001010110010101101011.info/ 2015-05-21 12:59:02 INFO fetched via newnzab. Retrying the send. 2015-05-21 12:59:02 INFO --Referencing : newznab @ 2015-05-20 21:46:42 2015-05-21 12:59:02 INFO Located snatched download: 2015-05-21 12:59:02 INFO Retrying : 489315 2015-05-21 12:59:02 INFO ComicID:72417

evilhero commented 9 years ago

Not nzbgeek, it's the Retry option - it's currently broken and will throw that error. It's next on the fix list.

Try downloading normally (don't use the retry option) and see if that works.

DGMayor commented 9 years ago

That was it!