search

evilhero / mylar

An automated Comic Book downloader (cbr/cbz) for use with SABnzbd, NZBGet and torrents
GNU General Public License v3.0
978 stars 173 forks source link

ValueError: invalid literal for int() with base 10: '' #485

Closed Rumtum closed 7 years ago

Rumtum commented 11 years ago

Dev Build: 16ddfad398627931c42a8162cc5fdef65e6ba011 Windows 7

Is this the custom exceptions problem mentioned in another recent issue?

31-Jul-2013 11:47:34 - INFO :: Thread-12 : Now adding/updating issues for Morning Glories 31-Jul-2013 11:47:41 - INFO :: Thread-12 : Updating complete for: Morning Glories 31-Jul-2013 11:47:41 - INFO :: Thread-12 : Now checking files for Morning Glories (2010) in r:\eComics\Morning Glories Exception in thread Thread-12: Traceback (most recent call last): File "C:\Program Files\Python27\lib\threading.py", line 808, in bootstrap_inner self.run() File "C:\Program Files\Python27\lib\threading.py", line 761, in run self.__target(_self.args, _self.__kwargs) File "C:\Program Files (x86)\Mylar\mylar\weeklypull.py", line 142, in pullit pullitcheck() File "C:\Program Files (x86)\Mylar\mylar\weeklypull.py", line 556, in pullitcheck statusupdate = updater.upcoming_update(ComicID=ComicID, ComicName=ComicName, IssueNumber=ComicIssue, IssueDate=Comic Date, forcecheck=forcecheck) File "C:\Program Files (x86)\Mylar\mylar\updater.py", line 179, in upcoming_update else: mylar.importer.addComictoDB(ComicID,mismatch,pullupd) File "C:\Program Files (x86)\Mylar\mylar\importer.py", line 761, in addComictoDB updater.forceRescan(comicid) File "C:\Program Files (x86)\Mylar\mylar\updater.py", line 525, in forceRescan fcdigit = helpers.issuedigits(fcredone[0]) File "C:\Program Files (x86)\Mylar\mylar\helpers.py", line 686, in issuedigits int_issnum = (int(issno) \ 1000) + ordtot ValueError: invalid literal for int() with base 10: ''

evilhero commented 11 years ago

It looks like something different - the easy way to tell is in the comicdetails url for the series, if the comicid starts with a G, then it's using the custom_exceptions, but considering it's calling the line it's calling, that's CV-related data...I'm just not sure why it's throwing the error. Do you have any files that exist in that directory that's it checking, and if so are the filenames all the same ? It looks like it's chunking up on the filename for some reason,but without seeing how it's named - I can't duplicate the problem on my end with the exact series. :(

Edit: oohhh, ok I think I see the problem - let me try to commit it up for ya to test, since I can't duplicate it.

Rumtum commented 11 years ago

This is still happening, but I am unable to reproduce on demand. I'll monitor things.

evilhero commented 11 years ago

You'll get the invalid literal when the filechecker parses the filename incorrectly and assigns a non-numeric string to the digit value...and it can't take the integer value of a string (especially a space)...if you get the error and can reproduce it, if you could provide the series, and the name of the filename it helps out immensely. With the new filechecker, as well as some new logic (in order to try and be proactive with alphanumeric issue numbering) things are abit of a mess atm :(

Rumtum commented 11 years ago

Windows using development branch c151f8eb0513b19c67b58de272f593b8945eec05

I can reproduce it again on recheck files.

The file in is R:\eComics\Superman Earth One\Superman - Earth One Vol One.cbz

Log file: 16-Aug-2013 14:32:00 - INFO :: Thread-35 : Now checking files for Superman: Earth One (2010) in r:\eComics\Superman Earth One 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : comic: Superman: Earth One 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : Looking in: r:\eComics\Superman Earth One 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : volume indicator detected as version #:Vol 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : subname no brackets: Superman - Earth One One.cbz 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : watchcomic:Superman Earth One ..comparing to found file: Superman Earth One One cbz 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : Superman Earth One - watchlist match on : r:\eComics\Superman Earth One\Superman - Earth One Vol One.cbz 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : we should remove 1 characters 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : detected 1 special characters 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : special character appears outside of title - ignoring @ position: 9 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : nonocount [0] cchk [Superman Earth One] length [18] 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : final jtd_len to prune [18] 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : before title removed from FILENAME [Superman - Earth One Vol One.cbz] 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : after title removed from FILENAME [ne Vol One.cbz] 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : creating just the digits using SUBNAME, pruning first [18] chars from [Superman Earth One One cbz] 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : after title removed from SUBNAME [One cbz] 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : JUSTTHEDIGITS [One] 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : final justthedigits [One] 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : you have a total of 1 Superman: Earth One comics 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : temploc: One 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : tstord: One 16-Aug-2013 14:32:00 - DEBUG :: Thread-35 : issno:

Console: 16-Aug-2013 14:32:00 - INFO :: Thread-35 : Now checking files for Superman: Earth One (2010) in r:\eComics\Superman Earth One Exception in thread Thread-35: Traceback (most recent call last): File "C:\Program Files\Python27\lib\threading.py", line 808, in bootstrap_inner self.run() File "C:\Program Files\Python27\lib\threading.py", line 761, in run self.__target(_self.args, _self.__kwargs) File "C:\Program Files (x86)\Mylar\mylar\updater.py", line 516, in forceRescan fcdigit = helpers.issuedigits(fcnew[som]) File "C:\Program Files (x86)\Mylar\mylar\helpers.py", line 696, in issuedigits int_issnum = (int(issno) \ 1000) + ordtot ValueError: invalid literal for int() with base 10: ''

If I change the file name to R:\eComics\Superman Earth One\Superman - Earth One Vol1.cbz, I get a different error.

Log: 16-Aug-2013 14:37:30 - INFO :: Thread-36 : Now checking files for Superman: Earth One (2010) in r:\eComics\Superman Earth One 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : comic: Superman: Earth One 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : Looking in: r:\eComics\Superman Earth One 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : volume indicator detected as version #:Vol1.cbz 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : subname no brackets: Superman - Earth One 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : watchcomic:Superman Earth One ..comparing to found file: Superman Earth One 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : Superman Earth One - watchlist match on : r:\eComics\Superman Earth One\Superman - Earth One Vol1.cbz 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : we should remove 1 characters 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : detected 1 special characters 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : special character appears outside of title - ignoring @ position: 9 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : nonocount [0] cchk [Superman Earth One] length [18] 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : final jtd_len to prune [18] 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : before title removed from FILENAME [Superman - Earth One Vol1.cbz] 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : after title removed from FILENAME [ne Vol1.cbz] 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : creating just the digits using SUBNAME, pruning first [18] chars from [Superman Earth One] 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : after title removed from SUBNAME [] 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : JUSTTHEDIGITS [] 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : final justthedigits [] 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : you have a total of 1 Superman: Earth One comics 16-Aug-2013 14:37:30 - DEBUG :: Thread-36 : temploc:

Console: 16-Aug-2013 14:37:30 - INFO :: Thread-36 : Now checking files for Superman: Earth One (2010) in r:\eComics\Superman Eart h One Exception in thread Thread-36: Traceback (most recent call last): File "C:\Program Files\Python27\lib\threading.py", line 808, in bootstrap_inner self.run() File "C:\Program Files\Python27\lib\threading.py", line 761, in run self.__target(_self.args, *_self.__kwargs) File "C:\Program Files (x86)\Mylar\mylar\updater.py", line 462, in forceRescan fcnew = shlex.split(fcnew_af[0]) IndexError: list index out of range

evilhero commented 11 years ago

Yeah graphic novels in general are a completely different beast because of the different numbering they use (some use Volume 1, 2, etc, some use actual numbers ala issue numbering, and others don't have any numbering). - right now the Volume catcher in the filechecker just checks for v# or Vxxxx or Volxxxx(where xxxx is the series year) and then removes it from the string so it can get a clean name. That's the reason why in your last example it threw the index out of range, just above that you can see that in the finaljustthedigits it removed everything (instead of leaving just the number).

If you were to change the filenames to 'Superman Earth One 1.cbz' it will work as if you look at the series within Mylar it actually has issue numbering. This doesn't get around the actual problem if you want to retain the 'Superman Earth One Volume One' kinda thing. I can make it recognize the One, Two, etc but it might get a bit sketchy trying to match up correctly because it's going from alpha to numerics when doing comparisons.

If most graphic novels have the Volume One, Volume Two (or something similar - Vol. One, etc) - I may be able to build in some exceptions into it,

Rumtum commented 11 years ago

I'll just delete the comic from the library, no problem.

I'll continue to monitor for standard comics.

ForumFerret commented 11 years ago

Got it while doing a general update (since I noticed some of my Pull List comics weren't marked "Wanted" in the weekly review).

27-Aug-2013 11:57:14 - INFO :: Thread-507 : Now adding/updating: Sex
27-Aug-2013 11:57:14 - INFO :: Thread-507 : Sucessfully retrieved details for Sex
27-Aug-2013 11:57:14 - INFO :: Thread-507 : Directory (/Volumes/WD-External/Comics/) already exists! Continuing...
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Sucessfully retrieved cover for Sex
i:80
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Sucessfully retrieved issue details for Sex
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Now adding/updating issues for Sex
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Updating complete for: Sex
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Now checking files for Sex (2013) in /Volumes/WD-External/Comics/
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Updated the status of 6 issues for Sex (2013) that were not found.
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Total files located: 0
27-Aug-2013 11:57:15 - INFO :: Thread-507 : I have physically found 0 issues, ignored 0 issues, and accounted for 5 in an Archived state. Total Issue Count: 5 / 6
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Checking this week pullist for new issues of Sex
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Checking the Weekly Releases list for comics I'm watching...
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Finished checking for comics on my watchlist.
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Attempting to grab wanted issues for : Sex
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Shhh be very quiet...I'm looking for Sex issue: 6 (2013) using experimental [RSS]
27-Aug-2013 11:57:15 - INFO :: Thread-507 : More than one search provider given - trying next one.
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Shhh be very quiet...I'm looking for Sex issue: 6 (2013) using nzb.su [RSS]
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Couldn't find Issue 6 of Sex(2013). Status kept as wanted.
27-Aug-2013 11:57:15 - INFO :: Thread-507 : Shhh be very quiet...I'm looking for Sex issue: 6 (2013) using experimental
Exception in thread Thread-507:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/patrickburke/Applications/Mylar/mylar/mylar/updater.py", line 100, in dbUpdate
    mylar.importer.addComictoDB(comicid,mismatch)
  File "/Users/patrickburke/Applications/Mylar/mylar/mylar/importer.py", line 789, in addComictoDB
    search.searchforissue(result['IssueID'])
  File "/Users/patrickburke/Applications/Mylar/mylar/mylar/search.py", line 1290, in searchforissue
    foundNZB, prov = search_init(result['ComicName'], result['Issue_Number'], str(IssueYear), comic['ComicYear'], IssueDate, result['IssueID'], AlternateSearch, UseFuzzy, ComicVersion, mode=mode, ComicID=ComicID)
  File "/Users/patrickburke/Applications/Mylar/mylar/mylar/search.py", line 274, in search_init
    findit = NZB_SEARCH(ComicName, IssueNumber, ComicYear, SeriesYear, nzbprov, nzbpr, IssDateFix, IssueID, UseFuzzy, newznab_host, ComicVersion, SARC=SARC, IssueArcID=IssueArcID, ComicID=ComicID)
  File "/Users/patrickburke/Applications/Mylar/mylar/mylar/search.py", line 990, in NZB_SEARCH
    comintIss = helpers.issuedigits(comic_iss)
  File "/Users/patrickburke/Applications/Mylar/mylar/mylar/helpers.py", line 696, in issuedigits
    int_issnum = (int(issno) * 1000) + ordtot
ValueError: invalid literal for int() with base 10: ''
evilhero commented 11 years ago

Ugh that's an invalid search result it's hanging on (experimental search) .. I'll look into it and see what's exactly going on.

As far as not being marked as 'Wanted', once the error was hit Mylar wouldn't continue looking thru the pullist for additional series that have new issues. It would just stop. A restart or shutdown/startup would continue the process for the next series and avoid the one giving the error (for this check anyways.. 5+hrs and it would throw it again probably).

Rumtum commented 10 years ago

Haven't been able to reproduce. Happy to close.