search

Nandaka / PixivUtil2

Download images from Pixiv and more!
http://nandaka.devnull.zone/
BSD 2-Clause "Simplified" License
2.4k stars 254 forks source link

Error at processImage() for non english folders #7

Closed Owyn closed 12 years ago

Owyn commented 12 years ago

PixivDownloader2 version 20120704 https://nandaka.wordpress.com/tag/pixiv-downloader/ Reading config file... done. Creating database... done. Only process member where day last updated >= 7 Using Username: test56 logging in with saved cookie Trying to log with saved cookie Cookie already expired/invalid. Log in using form. done. new cookie value: ad9248ee374adac1b63e855ce01f679f Writing config file... done. PixivDownloader2 version 20120704 https://nandaka.wordpress.com/tag/pixiv-downloader/

  1. Download by member_id
  2. Download by image_id
  3. Download by tags
  4. Download from list
  5. Download from online user bookmark
  6. Download from online image bookmark
  7. Download from tags list
  8. Download new illust from bookmark
  9. Download by Title/Caption

    10. Download by Tag and Member Id

d. Manage database e. Export online bookmark x. Exit Input: 1 Member id: 1598540 Start Page (default=1): End Page (default=0, 0 for no limit): Processing Member Id: 1598540 Reading config file... done. Page 1 Member Name : DanteWontDie Member Avatar: http://i2.pixiv.net/img46/profile/dantewontdie/4706570.jpg Member Token : dantewontdie

1

Processing Image Id: 28628322 Title: Want some fun? Tags : ???? ???? ??? ?????? ?? Mode : big Image URL : http://i2.pixiv.net/img46/img/dantewontdie/28628322.jpg Traceback (most recent call last): File "PixivUtil2.py", line 649, in processImage File "PixivHelper.pyc", line 49, in sanitizeFilename UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 32: ordinal not in range(128) Error at processImage(): (<type 'exceptions.UnicodeDecodeError'>, UnicodeDecodeE rror('ascii', 'V:\Documents and Settings\Admin\xd0\xe0\xe1\xee\xf7\xe8\xe9 \ xf1\xf2\xee\xeb\DL Image Packs\', 32, 33, 'ordinal not in range(128)'), <trace back object at 0x016A67D8>) Dumping html to: Error Medium Page for image 28628322.html Cannot dump page for image_id: 28628322 Stuff happened, trying again after 2 second ( 1 ) local variable 'parseBigImage' referenced before assignment Processing Image Id: 28628322 Title: Want some fun? Tags : ???? ???? ??? ?????? ?? Mode : big Image URL : http://i2.pixiv.net/img46/img/dantewontdie/28628322.jpg Traceback (most recent call last): File "PixivUtil2.py", line 649, in processImage File "PixivHelper.pyc", line 49, in sanitizeFilename UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 32: ordinal not in range(128) Error at processImage(): (<type 'exceptions.UnicodeDecodeError'>, UnicodeDecodeE rror('ascii', 'V:\Documents and Settings\Admin\xd0\xe0\xe1\xee\xf7\xe8\xe9 \ xf1\xf2\xee\xeb\DL Image Packs\', 32, 33, 'ordinal not in range(128)'), <trace back object at 0x01787198>) Dumping html to: Error Medium Page for image 28628322.html Cannot dump page for image_id: 28628322 Stuff happened, trying again after 2 second ( 2 ) local variable 'parseBigImage' referenced before assignment Processing Image Id: 28628322 Title: Want some fun? Tags : ???? ???? ??? ?????? ?? Mode : big Image URL : http://i2.pixiv.net/img46/img/dantewontdie/28628322.jpg Traceback (most recent call last): File "PixivUtil2.py", line 649, in processImage File "PixivHelper.pyc", line 49, in sanitizeFilename UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 32: ordinal not in range(128) Error at processImage(): (<type 'exceptions.UnicodeDecodeError'>, UnicodeDecodeE rror('ascii', 'V:\Documents and Settings\Admin\xd0\xe0\xe1\xee\xf7\xe8\xe9 \ xf1\xf2\xee\xeb\DL Image Packs\', 32, 33, 'ordinal not in range(128)'), <trace back object at 0x016BDA58>) Dumping html to: Error Medium Page for image 28628322.html Cannot dump page for image_id: 28628322 Stuff happened, trying again after 2 second ( 3 ) local variable 'parseBigImage' referenced before assignment Processing Image Id: 28628322 Title: Want some fun? Tags : ???? ???? ??? ?????? ?? Mode : big Image URL : http://i2.pixiv.net/img46/img/dantewontdie/28628322.jpg Traceback (most recent call last): File "PixivUtil2.py", line 649, in processImage File "PixivHelper.pyc", line 49, in sanitizeFilename UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 32: ordinal not in range(128) Error at processImage(): (<type 'exceptions.UnicodeDecodeError'>, UnicodeDecodeE rror('ascii', 'V:\Documents and Settings\Admin\xd0\xe0\xe1\xee\xf7\xe8\xe9 \ xf1\xf2\xee\xeb\DL Image Packs\', 32, 33, 'ordinal not in range(128)'), <trace back object at 0x01904788>) Dumping html to: Error Medium Page for image 28628322.html Cannot dump page for image_id: 28628322 Stuff happened, trying again after 2 second ( 4 ) local variable 'parseBigImage' referenced before assignment Processing Image Id: 28628322 Title: Want some fun? Tags : ???? ???? ??? ?????? ?? Mode : big Image URL : http://i2.pixiv.net/img46/img/dantewontdie/28628322.jpg Traceback (most recent call last): File "PixivUtil2.py", line 649, in processImage File "PixivHelper.pyc", line 49, in sanitizeFilename UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 32: ordinal not in range(128) Error at processImage(): (<type 'exceptions.UnicodeDecodeError'>, UnicodeDecodeE rror('ascii', 'V:\Documents and Settings\Admin\xd0\xe0\xe1\xee\xf7\xe8\xe9 \ xf1\xf2\xee\xeb\DL Image Packs\', 32, 33, 'ordinal not in range(128)'), <trace back object at 0x019081C0>) Dumping html to: Error Medium Page for image 28628322.html Cannot dump page for image_id: 28628322 Giving up image_id: 28628322 PixivDownloader2 version 20120704 https://nandaka.wordpress.com/tag/pixiv-downloader/

  1. Download by member_id
  2. Download by image_id
  3. Download by tags
  4. Download from list
  5. Download from online user bookmark
  6. Download from online image bookmark
  7. Download from tags list
  8. Download new illust from bookmark
  9. Download by Title/Caption

    10. Download by Tag and Member Id

d. Manage database e. Export online bookmark x. Exit Input:

Nandaka commented 12 years ago

I is working in my pc, try to set the regional settings to Japanese in control panel ==> http://imgur.com/etKuK

Owyn commented 12 years ago

But my system isn't japanese, it's russian

Nandaka commented 12 years ago

Is it only affecting that member_id? can you download from different member? Can you os reading any file with japanese word? The problem is cause by filename generation, can I get the filenameFormat?

my WinXP is set up with japanese for non-unicode program, or you can use AppLocale from microsoft to change the locale only for the downloader.

Owyn commented 12 years ago

Every download, no. Yes, os can read japanese named files. Filename generation is filenameformat = %member_id% (%member_token%)\%image_id%

it is because I did set output path to my desktop and 'desktop' is in russian, so with this bug I can't save to desktop

Nandaka commented 12 years ago

So the problem because it cannot read russian charset and it is related to #6. Can you try to create a shortcut with proper Start In folder? or you can create a batch file (runPixiv.bat) to change the dir and execute the dowloader, for example:

cd "c:\the application folder"
PixivUtil2.exe
Owyn commented 12 years ago

Application folder is correct, I don't think it's related to #6, just rootdirectory in config.ini is set to desktop which path includes russian symbols.

Nandaka commented 12 years ago

save config.ini in utf-8, don't use ansi in notepad

Owyn commented 12 years ago

yes, that's it. But I only edited the config file, it was created as ANSI. Maybe you could create it as utf-8 without bom by default with your program?

Nandaka commented 12 years ago

if no BOM, notepad will save it back to ANSI, I think I have added detector to skip BOM for the config file.

Nandaka commented 12 years ago

limitation of Config parser in python 2.7, No plan to fix. Please save the application on folder without special character.