Titles From the Pull List can't find issues

[ressiv]

Try to add "Avenging Spider-Man" from either the Pull list or the direct search box.

It is found initially, but won't get issue information.

12-Sep-2012 21:11:20 - INFO :: Thread-15 : Now adding/updating: Avenging Spider-Man 12-Sep-2012 21:11:25 - WARNING :: Thread-15 : No matching result found for Avenging Spider-Man (2011)

A few other series with the same 'spinning' behavior that never completes and the same log error: "New Avengers", "AvX VS", "Fantastic Four", "Witchblade"

[evilhero]

Yep you've found a couple doozies there. 4 separate bugs actually, but somewhat inner related. Here's the breakdown:

The Pullist doesn't like to do one-off issues at the moment, it's how the data is stored/retrieved that's causing the problems at the moment, but I noticed this one a few days ago.

The Avenging Spider-man, New Avengers, and Witchblade errors are all being caused because it can't match the records in the 2 databases to confirm what it is. This has to do with one db not updating their issue count as quickly as the other, or if the pub dates rollover between 2011/2012 (like for New Avengers), the series years are wrong entirely.

AvX VS is simply because of : in the search string that gets passed - I should be able to fix that one easy enough.

Fantastic Four is just a monster from the looks of it...it can't match because the records are entirely different. CV has it listed as 11 issues, gcd at 12 and then the whole mess of continuation of issue numbers...definitely gonna have to look into it more.

I might just have to recreate the entire code from scratch because of all these inconsistencies and try a different approach.

[evilhero]

latest commit should fix the issue of not being able to add AvX VS, as well as Avenging Spider-man (at least temporarily until a better fix) although Witchblade, New Avengers and FF still don't work which is dealing with issue counts and discrepancies.

[ressiv]

Great stuff - really enjoying your efforts! I wonder if some 'fuzzy' logic around the version info might allow it to proceed in the absence of an exact match, or conversely, present feedback on the discrepancy and let the user choose to accept the difference or choose a different series as the case may be? If an exact match is not going to be possible in all cases, which it sounds like it won't, that may be a tough target to reach for.

I only use ComicVine (and I assume most people have one primary authority that they use), so the extra validation is a bit of a mystery to me in terms of the value it adds to the process?

[evilhero]

Thanks muchly - glad some people are attempting to use this thing :)

It does use ComicVine (like you already guessed probably), and then attempts to validate the comicname, comicyear and total # of issues against gcd. It does this because it needs to be able to grab the publication date, and cv's api doesn't allow for a clean way to grab it. Like for the Walking Dead, for example, it has 101 issues. I'd have to do 101 seperate api calls to CV in order to get the issue publication date, or I can do one web call to gcd and parse the results :) It needs the issue publication date in order to get exact search results when looking or else it would pull down every issue for a series regardless of the version. CV also has some other issues, like the Walking Dead states 100 issues - but GCD states 101 issues (correctly). Witchblade is still a real work, as the difference between CV and GCD is 2 issues (GCD states 2 less than CV for some reason).

I'm working on some differential logic around the parsing and the way it compares stuff - right now I finally figured out how to get it to not bomb when adding a series and issue #'s aren't the same and some adjustments if it's out by an issue.

I'd like to get it all fuzzy , so there's as little intervention by the user once they say they want to add a series. But feedback on the discrepancy is definitly an alternative, although if it's the issues that are causing the problem it might cause Mylar to bork. So there's other logic therein - I might start to toss around the idea of the discrepancy if there's no match.

Any other ideas are most welcome...I'm still new to all of this stuff myself :)

[ressiv]

Ah I see... Merging two partial data sets to get a composite - that makes sense now! Thanks for the detailed explanation and thanks for putting all this together. This a cool new toy and I'm happy to spend some time helping in some small way. If only I had some time to familiarize myself with python enough to be of some help in the actual fixing.

I had forwarded what I hoped were some atypical series - they probably don't cover all of the possible permutations, but are hopefully representative of what else is out there in terms of volumes, years, issue counts, etc.

Something seemed wonky with github tonight, so I started fresh with the latest commit and a fresh db. [Windows 7, python 2.7.3]

Tested the following with both experimental and 'not' experimental:

Avenging Spider-man [works: check]

AVX : VS sadly resulted in a new error:

13-Sep-2012 20:16:01 - INFO :: Thread-12 : Now adding/updating: AVX: VS 13-Sep-2012 20:16:07 - INFO :: Thread-12 : Sucessfully retrieved details for AVX : VS Exception in thread Thread-12: Traceback (most recent call last): File "C:\Python27\lib\threading.py", line 551, in bootstrap_inner self.run() File "C:\Python27\lib\threading.py", line 504, in run self.__target(_self.args, *_self.__kwargs) File "C:\Users\John\Documents\GitHub\mylar\mylar\importer.py", line 112, in ad dComictoDB except OSError.e: AttributeError: type object 'exceptions.OSError' has no attribute 'e'

And one that was working before [Venom (2011)] is not:

13-Sep-2012 20:53:03 - INFO :: Thread-15 : Now adding/updating: Venom 13-Sep-2012 20:53:11 - INFO :: Thread-15 : Sucessfully retrieved details for Ven om 13-Sep-2012 20:53:11 - INFO :: Thread-15 : Directory (D:\CZFolder/Venom_(2011)) already exists! Continuing... 13-Sep-2012 20:53:11 - INFO :: Thread-15 : Sucessfully retrieved issue details f or Venom 13-Sep-2012 20:53:11 - INFO :: Thread-15 : Now adding/updating issues forVenom 13-Sep-2012 20:53:11 - INFO :: Thread-15 : Checking directory for existing issue s. 13-Sep-2012 20:53:11 - INFO :: Thread-15 : Found 0/27 issues of Venom Exception in thread Thread-15: Traceback (most recent call last): File "C:\Python27\lib\threading.py", line 551, in bootstrap_inner self.run() File "C:\Python27\lib\threading.py", line 504, in run self.__target(_self.args, _self.__kwargs) File "C:\Users\John\Documents\GitHub\mylar\mylar\importer.py", line 202, in ad dComictoDB gcdis = (int(issb4dec) \ 1000) + decisval ValueError: invalid literal for int() with base 10: '[nn]'

[evilhero]

2012-09-13 22:43:18     INFO    Finished grabbing what I could.
2012-09-13 22:43:18     INFO    No issues marked as wanted for Venom
2012-09-13 22:43:18     INFO    Updating complete for: Venom
2012-09-13 22:43:12     INFO    Found 0/28 issues of Venom
2012-09-13 22:43:12     INFO    Checking directory for existing issues.
2012-09-13 22:43:12     INFO    Now adding/updating issues forVenom
2012-09-13 22:43:12     INFO    Sucessfully retrieved issue details for Venom
2012-09-13 22:43:11     INFO    Directory successfully created at: /mount/mediavg/Comics/Comics/Venom_(2011)
2012-09-13 22:43:11     INFO    Sucessfully retrieved details for Venom
2012-09-13 22:42:59     INFO    Now adding/updating: Venom

Venom worked for me...I just updated the latest commit, so you might want to make sure you're on the same. I'm using Python 2.7.2 as well, which I didn't think was an issue since you're on 2.7.x.

I'll look into the avx problem..usually that error indicates that there's no directory in your Configuration, although at this point I'm guessing that the ':' might be causing directory creation problems.

Cheers for the help in finding bugs! :)

[ressiv]

Yup - that did it. Bypass github and grab lastest commit.

Yes, Windows doesn't like : in a file/directory name!

Better success now with new series loading! Thanks!