twiddli / happypanda

A cross platform manga/doujinshi manager with namespace & tag support
http://pewpew.moe/project/happypanda
Other
168 stars 32 forks source link

'It was too big' failure when import large galleries, and foldernames with " ' "9 #170

Closed emmauss closed 7 years ago

emmauss commented 7 years ago

this occur when i try to import hundreds of galleries(>900) . also when i try importing single galleries with a single-quote in their name, it also fails. log here,

happypanda.zip

also traceback from cmd

C:\Users\Emmanuel Hansen\Downloads\Programs\happypanda-alpha\version>python main.py Traceback (most recent call last): File "C:\Users\Emmanuel Hansen\Downloads\Programs\happypanda-alpha\version\misc.py", line 1226, in send_to_lib self.parent_widget.default_manga_view.add_gallery(galleries) File "C:\Users\Emmanuel Hansen\Downloads\Programs\happypanda-alpha\version\gallery.py", line 1631, in add_gallery Executors.generate_thumbnail(g, on_method=g.set_profile) NameError: name 'Executors' is not defined Exception in thread Method Queue Thread: Traceback (most recent call last): File "C:\Program Files\Python35\lib\threading.py", line 914, in _bootstrap_inner self.run() File "C:\Program Files\Python35\lib\threading.py", line 862, in run self._target(*self._args, *self._kwargs) File "C:\Users\Emmanuel Hansen\Downloads\Programs\happypanda-alpha\version\gallerydb.py", line 92, in process_methods r = method(args) File "C:\Users\Emmanuel Hansen\Downloads\Programs\happypanda-alpha\version\gallerydb.py", line 470, in add_gallery ChapterDB.add_chapters(object) File "C:\Users\Emmanuel Hansen\Downloads\Programs\happypanda-alpha\version\gallerydb.py", line 594, in add_chapters cls.executemany(cls, 'INSERT INTO chapters VALUES(NULL, ?, ?, ?, ?, ?, ?)', executing) File "C:\Users\Emmanuel Hansen\Downloads\Programs\happypanda-alpha\version\database\db.py", line 452, in executemany return self._DB_CONN.executemany(*args) sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 0, and there are 6 supplied.

emmauss commented 7 years ago

fails on both alpha and master.

twiddli commented 7 years ago

Hello, thanks for reporting.

I see multiple issues here. First, one of your files were causing some issues, namely: D:\1\812883-[ShindoL] Junai-Irregulars [English]\.jpg .jpg is invalid. To fix it, I will try to make it avoid those kind of files.

Second issue occurs when trying to communicate with the DB, I reckon it being related to #173 Is it possible for you to test it out again after merging that PR?

emmauss commented 7 years ago

i cleaned up the galleries, merge the PR, but then it fails for another reason. change in log: log file(end of file): `26-01 02:50 CRITICAL app File "C:\Users\Emmanuel Hansen\Downloads\Programs\pew\happypanda\version\misc.py", line 1226, in send_to_lib self.parent_widget.default_manga_view.add_gallery(galleries) File "C:\Users\Emmanuel Hansen\Downloads\Programs\pew\happypanda\version\gallery.py", line 1631, in add_gallery Executors.generate_thumbnail(g, on_method=g.set_profile)

26-01 02:50 CRITICAL app <class 'NameError'>: name 'Executors' is not defined 26-01 02:50 INFO app Analyzing database... 26-01 02:50 INFO app Closing database... cmd: Exception in thread Method Queue Thread: Traceback (most recent call last): File "C:\Program Files\Python35\lib\threading.py", line 914, in _bootstrap_inner self.run() File "C:\Program Files\Python35\lib\threading.py", line 862, in run self._target(*self._args, *self._kwargs) File "C:\Users\Emmanuel Hansen\Downloads\Programs\pew\happypanda\version\gallerydb.py", line 92, in process_methods r = method(args) File "C:\Users\Emmanuel Hansen\Downloads\Programs\pew\happypanda\version\gallerydb.py", line 463, in add_gallery cursor = cls.execute(cls, default_exec(object)) File "C:\Users\Emmanuel Hansen\Downloads\Programs\pew\happypanda\version\database\db.py", line 433, in execute return self._DB_CONN.execute(args) sqlite3.OperationalError: table series has no column named artist

Traceback (most recent call last): File "C:\Users\Emmanuel Hansen\Downloads\Programs\pew\happypanda\version\misc.py", line 1226, in send_to_lib self.parent_widget.default_manga_view.add_gallery(galleries) File "C:\Users\Emmanuel Hansen\Downloads\Programs\pew\happypanda\version\gallery.py", line 1631, in add_gallery Executors.generate_thumbnail(g, on_method=g.set_profile) NameError: name 'Executors' is not defined`

twiddli commented 7 years ago

Hi, I forgot to mention that for #173 to work you need a fresh install. Just delete the db folder.

Also sorry, for the NameError issue I pushed the fix into the wrong branch. I've fixed it now.

So basically:

  1. git pull (fetch the fix)
  2. delete the db folder
  3. try again
emmauss commented 7 years ago

okay it now works. will be closing it as one was caused buy invalid naming of files, and the other was fixed by the sql merge.