search

evilhero / mylar

An automated Comic Book downloader (cbr/cbz) for use with SABnzbd, NZBGet and torrents
GNU General Public License v3.0
976 stars 169 forks source link

500 internal server error: #66

Closed AtariBaby closed 11 years ago

AtariBaby commented 12 years ago

I thought I'd check to see if "The Boys" was searchable and downloadable yet. After clicking on "Add this comic":

500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

Traceback (most recent call last): File "/home/misterfantastic/.mylar/cherrypy/_cprequest.py", line 645, in respond response.body = self.handler() File "/home/misterfantastic/.mylar/cherrypy/lib/encoding.py", line 188, in call self.body = self.oldhandler(_args, _kwargs) File "/home/misterfantastic/.mylar/cherrypy/_cpdispatch.py", line 29, in call return self.callable(_self.args, _self.kwargs) File "/home/misterfantastic/.mylar/mylar/webserve.py", line 119, in addComic fakeit = parseit.GCDAdd(testspx) File "/home/misterfantastic/.mylar/mylar/parseit.py", line 421, in GCDAdd subpubst = pubst('a')[0] TypeError: 'NoneType' object is not callable

evilhero commented 12 years ago

Just tried both of The Boys series (2006, and 2007) - and both added fine with no issues. I'm going to guess that there was an error prior to this one, and is causing some grief. Shutdown and then restart Mylar and see if that fixes the problem.

AtariBaby commented 12 years ago

Still occurs.

AtariBaby commented 12 years ago

This doesn't occur on other comics I've searched for. Only these two special cases: The Boys and Fantastic Four.

evilhero commented 12 years ago

Are you using latest comit with exceptions.csv? It will say loading from exceptions when Mylar starts in the logs. Clicking on either of those titles in the searchresults (to add a series)should bring up the volume-spanning screen where you can pick the correct volume.

AtariBaby commented 12 years ago

24-Oct-2012 12:42:06 - INFO :: MainThread : Populating Base Exception listings into Mylar....

-Oct-2012 12:42:06 - ERROR :: MainThread : No Custom Exceptions found. Using base exceptions only

AtariBaby commented 12 years ago

Is there some config somewhere that needs to be edited, like the location of one of my folders or anything? Perhaps some file I should manually replace to make sure the latest commits take?

evilhero commented 12 years ago

Make sure your exceptions.csv hasn't been modified...aside from that when you do a search for fantastic four and pick the series that has 600+ issues, you should be brought into the volume spanning screen and the log should indicate that as well.

As an extreme, backup your mylar.db and config.ini and wipe out do a fresh git install, although you shouldn't have to do that (copy them back afterwards and everything will be as it is)

AtariBaby commented 12 years ago

Ah bummer. did the "extreme". same result.

I was 11 commits behind when I did this update. Is it possible there was a file or files in one of those commits that isn't in the latest version?

evilhero commented 12 years ago

I'd check the exceptions.csv file you have on your machine and see if it has the following : 

#-----Volume Spanning Series-----
#Fantastic Four
4,2045,none,1482/10251/6029/11218/62349
#Amazing Spider-man
2,2127,none,1570/7794/11288
#The Boys
1,18033,none,19531/25058
#--------
AtariBaby commented 12 years ago

-----Volume Spanning Series-----

Fantastic Four

4,2045,none,1482/10251/6029/11218/62349

Amazing Spider-man

2,2127,none,1570/7794/11288

The Boys

1,18033,none,19531/25058

--------

-----Mismatched Names------

99,3092,2605,none 99,50389,66832,none 99,42947,61242,none 99,51382,68125,none 99,51622,68322,none 99,4937,4611,none 99,7300,7880,none 99,46744,67757,none 99,42821,60934,none 99,42947,61242,none 99,42322,60917,none

AtariBaby commented 12 years ago

Could a third party try this out for us?

CyberCowboy commented 12 years ago

Atari, I can check, what specifically would you like me to try?

AtariBaby commented 12 years ago

Thanks, Cowboy! If you'll add Fantastic Four (last page of search results, starts in the 1960's) or The Boys (69 issues), that would be great. This will test evilhero's new feature. Others have already tried it and succeeded, but if you don't mind taking a few minutes, it might demonstrate whether or not I'm the only person it's failing on.

CyberCowboy commented 12 years ago

I did get the attached screen shot when adding the 1961's version of Fantastic Four

If I can do more let me know.

Thanks, Cowboy! If you'll add Fantastic Four (last page of search results, starts in the 1960's) or The Boys (69 issues), that would be great. This will test evilhero's new feature. Others have already tried it and succeeded, but if you don't mind taking a few minutes, it might demonstrate whether or not I'm the only person it's failing on.

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9816840

CyberCowboy commented 12 years ago

erm screenshot failed to attach, it can be viewed here https://dl.dropbox.com/u/204578/ScreenShot.png (give it a moment to upload)

AtariBaby commented 12 years ago

That appears to be a screenshot of Sabnzbd+ ?

AtariBaby commented 12 years ago

Not to be nosy but you have a very interesting collection of apps and webpages there :p

CyberCowboy commented 12 years ago

yea it was supposed to replace that but DropBox is being weird I'll throw it on another server and send a new link, but basically it seemed to work for me.

That appears to be a screenshot of Sabnzbd+ ?

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9817136

CyberCowboy commented 12 years ago

Yea the life of an IT worker you collect crap and never delete it.

anyway here's the correct screen shot http://bryanjam.es/files/ScreenShot.png

Not to be nosy but you have a very interesting collection of apps and webpages there :p

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9817195

evilhero commented 12 years ago

How are you starting Mylar? are you using any command line options (to specify a data-dir or the like)? What version of python are you running on ?

CyberCowboy commented 12 years ago

My guess is these questions are for Atari, but for comparison sake

I'm starting Mylar with a @reboot entry in crontab that calls "Mylar.py --daemon"

Python version is 2.6.6

system is Ubuntu 12.04 x64

How are you starting Mylar? are you using any command line options (to specify a data-dir or the like)? What version of python are you running on ?

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9817544

AtariBaby commented 12 years ago

Very grateful for the support guys.

I'm running Ubuntu 12.04 x64 also, with all the most recent updates. CLI "python --version" reveals 2.7.3

I have mylar running in "Startup Applications Preferences" with the command line "python /home/misterfantastic/.mylar/Mylar.py" but it also occurs if shutdown and I type same command in terminal.

Have deleted and reinstalled Mylar, though I did paste the previously existing config file and the database file as directed. I have also restarted it, updated to latest version, and rebooted the machine, and I still get the error in OP.

Is there possibly something about my machine that doesn't know how to handle csv files? Is it the fact I have a newer version of python than CyberCowboy? Just throwing random darts here. Still new to ubuntu and all this.

I also run Sickbeard, Sabnzbd+ and Couchpotato on this machine. Not that it matters but this machine downloads comics then everything is configured to move them to my dropbox account.

AtariBaby commented 12 years ago

Anything else I can do to help with this?

AtariBaby commented 12 years ago

Would it help if I seek help from python user forums, etc?

evilhero commented 12 years ago

Actually if anything CyberCowboy would be having more problems than you if it was a python-related issue. Mylar is programmed using Python 2.7.2, the 2.7.x series are more compatible within their own branch, than the 2.6.x running on a 2.7.x.

One other thing, you can try with a new db and see if the problem finds it's way still. Backup your existing mylar.db, delete it, then start up mylar and see if you're still getting the problems. If you don't get any errors, then it's some underlying sql statement that's probably causing the issue, although I wouldn't know exactly which one right at the moment.

AtariBaby commented 12 years ago

i have previously completely deleted the entire mylar directory and reinstalled to no avail. Think it would be worthwhile to try this?

evilhero commented 12 years ago

Well if you reinstalled, but put back your original mylar.db file - you'd just be moving back the problem (if that's where the problem lies..the sql)

AtariBaby commented 12 years ago

The issue occurred even without moving my config and db files back over.

AtariBaby commented 12 years ago

Oh, the pain, occurs on fresh install. Is there anything after a fresh install that must be configured that might cause this to occur? And let me know if I can help by seeking help in programmer forums, etc.

CyberCowboy commented 12 years ago

Do you have any other websites running on the box (sickbeard etc?) it almost sounds like a problem with the web server. I don't believe Mylar has a built in webserver (I've been wrong before though, just ask my wife) so perhaps something with your Apache config got screwed up. A 500 error would possibly support that. If you go to http://<mylar server ip address> directly without the :<port #> afterwards what do you get?

Oh, the pain, occurs on fresh install. Is there anything after a fresh install that must be configured that might cause this to occur? And let me know if I can help by seeking help in programmer forums, etc.

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9918264

CyberCowboy commented 12 years ago

This http://www.checkupdown.com/status/E500.html supports the thought that it's a problem with the webserver software rather than Mylar directly.

Oh, the pain, occurs on fresh install. Is there anything after a fresh install that must be configured that might cause this to occur? And let me know if I can help by seeking help in programmer forums, etc.

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9918264

AtariBaby commented 12 years ago

CyberCowboy, I do have many websites running: sickbeard, sabnzbd+, couch potato. All seem to be working fine but who knows. If I attempt to go to http://localhost on the machine itself, or if I try to go to http://10.0.1.4 I get page not found or cannot connect errors. any idea what I should do or try next?

CyberCowboy commented 12 years ago

Is there a base index.html page that should load? What system are you using? Also Evilhero, does mylar have a self contained Web server or use the systems?

AtariBaby notifications@github.com wrote:

CyberCowboy, I do have many websites running: sickbeard, sabnzbd+, couch potato. All seem to be working fine but who knows. If I attempt to go to http://localhost on the machine itself, or if I try to go to http://10.0.1.4 I get page not found or cannot connect errors. any idea what I should do or try next?

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9928914

Sent from my Android phone with K-9 Mail. Please excuse my brevity.

AtariBaby commented 12 years ago

Well, I think this is just whatever is built into Ubuntu 12.04. If i set up anything additional, it was from a guide long ago when I was just grasping Ubuntu (well, I'm still grasping, but yanno)

CyberCowboy commented 12 years ago

Ok gotta sleep now but make sure you have an index.html in /var/www and open it so you know what to look for, more help hopefully tomorrow

AtariBaby notifications@github.com wrote:

Well, I think this is just whatever is built into Ubuntu 12.04. If i set up anything additional, it was from a guide long ago when I was just grasping Ubuntu (well, I'm still grasping, but yanno)

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9930855

Sent from my Android phone with K-9 Mail. Please excuse my brevity.

AtariBaby commented 12 years ago

i have a var directory but not a www subdirectory. This is all surprising to me since as I say, all my other apps seem to work fine: sickbeard, sab, couch potato...

CyberCowboy commented 12 years ago

I'm not sure if the other apps (or mylar for that matter) use the native apache server, or have a self-contained web server.

Do: "sudo apt-get install apache2"

If it attempts to install something it wasn't installed and that is likely your problem, if it says "apache2 is already the newest version." or asks if you want to UPGRADE it was installed (then an upgrade is a good idea but likely won't fix your problem)

For your Ubuntu install, did you install the server or desktop version? And was 12.04 a clean install or did you upgrade from a previous version?

i have a var directory but not a www subdirectory. This is all surprising to me since as I say, all my other apps seem to work fine: sickbeard, sab, couch potato...

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9933807

CyberCowboy commented 12 years ago

just did a test on mine (stopped the apache2 service) and mylar continued to work. This tells us that there's a built in webserver in mylar that is likely causing the problem. My previous questions of did you install the ubuntu 12.04 fresh or upgrade from a previous version, and did you install the ubuntu server or ubuntu desktop version (I'm guessing desktop since you don't have /var/www) become more important.

I suspect the problem may be a missing or out of date library that is on most machines that we need to get squared away for you, the problem is going to be finding exactly which library. I'm by no means an expert, but I'll take you through as far as I can.

i have a var directory but not a www subdirectory. This is all surprising to me since as I say, all my other apps seem to work fine: sickbeard, sab, couch potato...

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9933807

evilhero commented 12 years ago

Mylar uses Cherrypy as it's webserver. It's possible to run it behind an apache (from what I read in the past), but that's beyond my knowledge.

The error you're getting is related to the BeautifulSoup module in python that's used to parse webpages. The problem may lie somewhere in that aspect since its only happening to you, and based on what I've seen of some of your previous posts that you're running a bunch of vm's(or have several different type of os'). I know when I've seen that error before it basically means it can't find the line it's trying to locate - so it can't find the first reference of an 'a href' within the page it calls up. But it's working for almost everyone else so it's a struggle to figure out exactly what's unique on your system vs. everyone else.

I'll see if I can throw in some debug code in there to catch an exception at that point without breaking anything else - might help get past that point for ya .

CyberCowboy commented 12 years ago

Also Atari, if you want to zip the whole folder mylar is in I can try it on my system. That can certainly isolate if it's a problem internal to mylar or a library/module.

NOTE: this will include your config and your login credentials to any/all sites you've configured in mylar so if you'd rather not send that I TOTALLY understand.

Mylar uses Cherrypy as it's webserver. It's possible to run it behind an apache (from what I read in the past), but that's beyond my knowledge.

The error you're getting is related to the BeautifulSoup module in python that's used to parse webpages. The problem may lie somewhere in that aspect since its only happening to you, and based on what I've seen of some of your previous posts that you're running a bunch of vm's(or have several different type of os'). I know when I've seen that error before it basically means it can't find the line it's trying to locate - so it can't find the first reference of a within the page it calls up. But it's working for almost everyone else so it's a struggle to figure out exactly what's unique on your system vs. everyone else.

I'll see if I can throw in some debug code in there to catch an exception at that point without breaking anything else - might help get past that point for ya .

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9948267

AtariBaby commented 12 years ago

Yeah, I feel bad it only seems to be happening to me. The setup is not that unusual in that it's only Ubuntu 12.04 but there are lots of sick beard -like operations going on. When I talk about OSX, I'm just accessing the web page from another computer, and reporting issues when looking on that system. No VM.

CyberCowboy I installed 12.04 desktop from scratch. It is fully updated.

Thank you both!

AtariBaby commented 12 years ago

Gladly. How do I send?

CyberCowboy commented 12 years ago

if you've created a zip file as long as the total size is under 10 mb you can simply attach it to an e-mail and send it to me at bjames (a) jamesgang (dot) dyndns (dot) org

If it's larger than 10 MB do you have a dropbox account that you could plop it into the public folder and shoot me a link to it (do it to the e-mail address above so no one else will have it and possibly download and use your logins)

Gladly. How do I send?

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9948660

evilhero commented 12 years ago

try newest commit - exception trap put in place for your error...might let you by-pass it now :)

AtariBaby commented 12 years ago

Cybercowboy: file sent by email!

evilhero: just tried it. Still occurs!

CyberCowboy commented 12 years ago

nothing yet, by my server is occasionally slow, if I don't get it by 1 EST I'll let you know otherwise assume all is good.

Cybercowboy: file sent by email!

evilhero: just tried it. Still occurs!

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9951608

CyberCowboy commented 12 years ago

Atari your setup is working as intended for me. I am however running an Ubuntu 12.04 server 64-bit edition

do the debug codes that evilhero included give you any new feedback?

Cybercowboy: file sent by email!

evilhero: just tried it. Still occurs!

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9951608

AtariBaby commented 12 years ago

Ah I didn't look at any logs will paste when able

Sent from my iPhone

On Oct 31, 2012, at 10:02 AM, CyberCowboy notifications@github.com wrote:

Atari your setup is working as intended for me. I am however running an Ubuntu 12.04 server 64-bit edition

do the debug codes that evilhero included give you any new feedback?

Cybercowboy: file sent by email!

evilhero: just tried it. Still occurs!

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9951608

— Reply to this email directly or view it on GitHub.

CyberCowboy commented 12 years ago

Just throwing some thoughts out there so feel free to ignore/answer any you feel relevent:

  1. How were Sickbeard, Couchpotato, Sab, etc installed? from source, from PPA, or some other way?
  2. How did you install mylar (git pull?)
  3. I'm wondering if installing apache2 might not resolve some sort of dependency, however it obviously takes some space, and starts some other services. You can always "sudo apt-get remove apache2" if it fails

BTW I obviously got the zipped file you sent me so remember to delete it from a public share so no one can grab your login info (I work in IT security is why I keep harping on that, it's in my blood)

Ah I didn't look at any logs will paste when able

Sent from my iPhone

On Oct 31, 2012, at 10:02 AM, CyberCowboy notifications@github.com wrote:

Atari your setup is working as intended for me. I am however running an Ubuntu 12.04 server 64-bit edition

do the debug codes that evilhero included give you any new feedback?

Cybercowboy: file sent by email!

evilhero: just tried it. Still occurs!

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9951608

— Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub: https://github.com/evilhero/mylar/issues/66#issuecomment-9953062

AtariBaby commented 12 years ago

Someone else reports a 500 error: http://comicrack.cyolito.com/forum/7-general/14368-sick-beard-for-comics--now-that-would-be-cool?limit=10&start=560#26777