search

Nandaka / PixivUtil2

Download images from Pixiv and more!
http://nandaka.devnull.zone/
BSD 2-Clause "Simplified" License
2.35k stars 258 forks source link

[Enhancement] cmd call get return with os.exit #70

Closed sabearcher closed 9 years ago

sabearcher commented 9 years ago

I did write a python shell to use this PixivUtil2.exe in windows cmd, like os.system('"D:\Program Files\pixivutil20150218" -x -s 2 '+picId). but in the return result of os.system(), I only get 0, so would you please pick out other exceptions to return -1 or -2 or more with a manural, handle problems like what I've got many times that: timeout happens and PixivUtil2 exits, prints stack error on the screen but exits with value 0. Waiting for your reply, best wishes.

sabearcher commented 9 years ago

maybe I didn't make it clear, I mean that, I want to get an errorcode return while current download unfinished.

Nandaka commented 9 years ago

try http://www.mediafire.com/download/uy8ovvcp1luru51/pixivutil20150321-beta2.7z

sabearcher commented 9 years ago

is there any way to test? I've no idea how a exception comes out. and by the way, when connect error or sth. else occurred like errcode in readme.txt, will it sys.exit(errcode)?

sabearcher commented 9 years ago

_TEST DATA 1_

res = os.system(command) PixivDownloader2 version 20150321-beta2 https://nandaka.wordpress.com/tag/pixiv-downloader/ Reading E:\tmp\pixivutil20150321-beta2\config.ini ... Error at loadConfig(): (<type 'exceptions.IOError'>, IOError(2, 'No such file or directory'), <traceback object at 0x02C53CB0>) Some configuration have invalid value, replacing with the default value. Writing config file... Backing up old config (error exist!) to config.ini.error- 1427383404 done. done. Creating database... done. Only process member where day last updated >= 7 Username ? 3 Password ? Log in using form. Server Reply: pixiv ID、またはメールアドレスが不正です Server Reply: パスワードは半角6文字以上32文字以下にしてください Server Reply: pixiv ID、またはメールアドレスが不正です Server Reply: パスワードは半角6文字以上32文字以下にしてください Traceback (most recent call last): File "PixivUtil2.py", line 1756, in File "PixivUtil2.py", line 1752, in main NameError: global name 'exit' is not defined res 255

_TEST DATA 2_ done. Creating database... done. Only process member where day last updated >= 7 Username ? my user id correct Password ?my password correct Log in using form. Logged in new cookie value: 5021835_5142c6c895538c9270fb8f073ecb466f Writing config file... done. Processing Image Id: 1 Image ID (1): 'NoneType' object has no attribute 'find' Dumping html to: Error medium page for image 1.html Traceback (most recent call last): File "PixivUtil2.py", line 1756, in File "PixivUtil2.py", line 1752, in main NameError: global name 'exit' is not defined

res 255

exit is a function of import(sys) and in TEST1 I should get 1001 , in TEST2 I should get 2006 , for other errors I should get other errcodes. ps: \================================================================================= = error codes = \=================================================================================

Nandaka commented 9 years ago

difference between sys.exit() to os._exit()? I haven't add the logic to get the actual error code from the exception :smile:

sabearcher commented 9 years ago

I do appologize for my misleading. what I got is _NameError: global name 'exit' is not defined_ so u must have declared _global exit_, that cause the 255 return.

I've tested these: import sys import os exit(17) os._exit(21) sys.exit(19)

each exit function returns correctly, truth there're some differences between the 3 functions, but just same to get a os.system(...) return

sabearcher commented 9 years ago

WHATEVER, i test with v20150328

  1. Download by member_id
  2. Download by image_id
  3. Download by tags
  4. Download from list
  5. Download from online user bookmark
  6. Download from online image bookmark
  7. Download from tags list
  8. Download new illust from bookmark
  9. Download by Title/Caption
  10. Download by Tag and Member Id
  11. Download Member Bookmark
  12. Download by Group Id

d. Manage database e. Export online bookmark r. Reload config.ini p. Print config.ini x. Exit Input: 2 Image ids: 1 Processing Image Id: 1 Image ID (1): 'NoneType' object has no attribute 'find' Dumping html to: Error medium page for image 1.html

os.system get return of 0, I've read your code, you've handled the download error with a except catch, seted the global ERROR_CODE=ex.errorCode, then return a PixivConstant.PIXIVUTIL_NOT_OK, it seems that exit will return the errorCode, but truthly not. I suggest u write a test to check if ERROR_CODE has been reseted -- my environment has some errors so i cannot run your py code directly :smirk:

Nandaka commented 9 years ago

Try: http://www.mediafire.com/download/n3tnmtxx8uiitw8/pixivutil20150403-beta.7z

sabearcher commented 9 years ago

examples

memberId errorcode right

"E:\tmp\pixivutil20150403-beta\PixivUtil2.exe" -x -s 1 1 PixivDownloader2 version 20150403-beta https://nandaka.wordpress.com/tag/pixiv-downloader/ Reading E:\tmp\pixivutil20150403-beta\config.ini ... done. .......... header: Set-Cookie: p_ab_id=5; expires=Fri, 10-Apr-2020 09:47:34 GMT; Max-Age=15 7852800; path=/; domain=.pixiv.net header: X-Frame-Options: SAMEORIGIN Login successfull. Processing Member Id: 1 Reading E:\tmp\pixivutil20150403-beta\config.ini ... done. Page 1 Member Url: http://www.pixiv.net/member_illust.php?id=1&p=1 send: 'GET /member_illust.php?id=1&p=1 HTTP/1.1\r\nAccept-Encoding: identity\r\n Referer: http://www.pixiv.net\r\nHost: www.pixiv.net\r\nCookie: module_orders_my page=%5B%7B%22name%22%3A%22everyone_new_illusts%22%2C%22visible%22%3Atrue%7D%2C% 7B%22name%22%3A%22spotlight%22%2C%22visible%22%3Atrue%7D%2C%7B%22name%22%3A%22fe atured_tags%22%2C%22visible%22%3Atrue%7D%2C%7B%22name%22%3A%22contests%22%2C%22v isible%22%3Atrue%7D%2C%7B%22name%22%3A%22following_new_illusts%22%2C%22visible%2 2%3Atrue%7D%2C%7B%22name%22%3A%22mypixiv_new_illusts%22%2C%22visible%22%3Atrue%7 D%2C%7B%22name%22%3A%22booth_follow_items%22%2C%22visible%22%3Atrue%7D%5D; pab id=5; PHPSESSID=5021835_0224a1caafb666ca8c779ba82869a0f5\r\nConnection: close\r\ nUser-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36\r\n\r\n' reply: 'HTTP/1.1 404 Not Found\r\n' header: Server: nginx header: Date: Fri, 10 Apr 2015 09:47:37 GMT header: Content-Type: text/html; charset=UTF-8 header: Transfer-Encoding: chunked header: Connection: close header: Vary: Accept-Encoding header: X-Host-Time: 64 header: Expires: Thu, 19 Nov 1981 08:52:00 GMT header: Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-ch eck=0 header: Pragma: no-cache Member ID (1): 1001 'User ID not exist/deleted!' Set IsDeleted for MemberId: 1 not exist.

res: 1001

_picture illustid errorcode wrong

"E:\tmp\pixivutil20150403-beta\PixivUtil2.exe" -x -s 2 1 PixivDownloader2 version 20150403-beta https://nandaka.wordpress.com/tag/pixiv-downloader/ Reading E:\tmp\pixivutil20150403-beta\config.ini ... done. Creating database... done. .......... Processing Image Id: 1 send: 'GET /member_illust.php?mode=medium&illust_id=1 HTTP/1.1\r\nAccept-Encodin g: identity\r\nReferer: http://www.pixiv.net\r\nHost: www.pixiv.net\r\nCookie: m odule_orders_mypage=%5B%7B%22name%22%3A%22everyone_new_illusts%22%2C%22visible%2 2%3Atrue%7D%2C%7B%22name%22%3A%22spotlight%22%2C%22visible%22%3Atrue%7D%2C%7B%22 name%22%3A%22featured_tags%22%2C%22visible%22%3Atrue%7D%2C%7B%22name%22%3A%22con tests%22%2C%22visible%22%3Atrue%7D%2C%7B%22name%22%3A%22following_new_illusts%22 %2C%22visible%22%3Atrue%7D%2C%7B%22name%22%3A%22mypixiv_new_illusts%22%2C%22visi ble%22%3Atrue%7D%2C%7B%22name%22%3A%22booth_follow_items%22%2C%22visible%22%3Atr ue%7D%5D; p_ab_id=2; PHPSESSID=5021835_0224a1caafb666ca8c779ba82869a0f5\r\nConne ction: close\r\nUser-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537. 36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36\r\n\r\n' reply: 'HTTP/1.1 404 Not Found\r\n' header: Server: nginx header: Date: Fri, 10 Apr 2015 09:50:30 GMT header: Content-Type: text/html; charset=UTF-8 header: Transfer-Encoding: chunked header: Connection: close header: Vary: Accept-Encoding header: X-Host-Time: 141 header: Expires: Thu, 19 Nov 1981 08:52:00 GMT header: Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-ch eck=0 header: Pragma: no-cache Image ID (1): 'NoneType' object has no attribute 'find' Dumping html to: Error medium page for image 1.html

res: 0

conclusion

please look at these: -Image ID (1): 'NoneType' object has no attribute 'find' -Dumping html to: Error medium page for image 1.html a function exception occurs, than errorcode of program not returns, errorcode lost after expect-catch

Nandaka commented 9 years ago

yap, cos generic exception doesn't have error code.

sabearcher commented 9 years ago

I hope for that, in any case the download of a query session don't finish, it might return a non-0 value for handling

sabearcher commented 9 years ago

very important case of return code

case 1

Trying to log with saved cookie Traceback (most recent call last): File "PixivUtil2.py", line 1712, in main File "PixivBrowserFactory.pyc", line 157, in loginUsingCookie File "mechanize_mechanize.pyc", line 203, in open File "mechanize_mechanize.pyc", line 230, in _mech_open File "mechanize_opener.pyc", line 193, in open File "mechanize_urllib2_fork.pyc", line 344, in _open File "mechanize_urllib2_fork.pyc", line 332, in _call_chain File "mechanize_urllib2_fork.pyc", line 1142, in http_open File "mechanize_urllib2_fork.pyc", line 1118, in do_open URLError: <urlopen error [Errno 10060] > res: 0

case 2

send: 'GET /mypage.php HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: www.pixiv. net\r\nCookie: PHPSESSID=5021835_fcea602e516f2adf6ed7ef260ea61252\r\nConnection: close\r\nUser-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KH TML, like Gecko) Chrome/37.0.2062.124 Safari/537.36\r\n\r\n' Traceback (most recent call last): File "PixivUtil2.py", line 1712, in main File "PixivBrowserFactory.pyc", line 157, in loginUsingCookie File "mechanize_mechanize.pyc", line 203, in open File "mechanize_mechanize.pyc", line 230, in _mech_open File "mechanize_opener.pyc", line 193, in open File "mechanize_urllib2_fork.pyc", line 344, in _open File "mechanize_urllib2_fork.pyc", line 332, in _call_chain File "mechanize_urllib2_fork.pyc", line 1142, in http_open File "mechanize_urllib2_fork.pyc", line 1118, in do_open URLError: res: 0

as you see, time out / urlopen error return res with 0, so my program will delete the pixivid ruled a mission completed, that's why I hope for a non-0 return code while exception occurs.