search

SpiderClub / weibospider

:zap: A distributed crawler for weibo, building with celery and requests.
MIT License
4.81k stars 1.21k forks source link

云打码平台好像失效了，之前那个超级鹰平台的issues下的temp_verification我按照操作来可是出了奇怪的bug，请问能根据新的打码平台更新一下吗，麻烦了 #207

Closed zjyzh closed 4 years ago

zjyzh commented 4 years ago

在提交Issue之前请先回答下面问题，谢谢！

1.你是怎么操作的？ 在linux环境下按照temp_verification项目进行操作，然后也替换掉了超级鹰的账号密码

尽量把你的操作过程描述清楚，最好能够复现问题。

然后利用 celery -A tasks.workers -Q login_queue worker -l info --concurrency=2 -Ofair 开始登陆业务，就出现了bug

2.你期望的结果是什么？ 能出一个稳定打码平台的版本吗，temp_verification项目分支我好像弄不了...

3.实际上你得到的结果是什么？

[2020-06-28 23:25:45,239: ERROR/ForkPoolWorker-1] Task tasks.login.login_task[0ae0f9d9-a689-4242-8072-43aa9ea29d7e] raised unexpected: UnicodeDecodeError('gbk', b'<!doctype html>\n<html>\n<head>\n <meta charset="utf-8">\n <meta http-equiv="X-UA-Compatibl" content="IE=edge,chrome=1"/>\n\n <title>\xe5\xbc\x82\xe5\xb8\xb8\xe8\xae\xbf\xe9\x97\xae\xe6\x8f\x90\xe7\xa4\xba</title>\n <link href="/css/denyerrorpage/frame.css" type="text/css" rel="stylesheet">\n <link href="/css/denyerrorpage/error.css" type="text/css" rel="stylesheet">\n <link href="/css/denyerrorpage/skin.css" type="text/css" rel="stylesheet">\n</head>\n<body>\n<div class="WB_miniblog">\n <!-- \xe9\xa6\x96\xe9\xa1\xb5 -->\n <div class="iforgot_bd">\n <div class="iforgot_header clearfix">\n <div class="logo_mod1 W_fl"></div>\n <div class="name_mod W_fr">\n <a href="http://www.sina.com.cn/" class="S_txt1">\xe6\x96\xb0\xe6\xb5\xaa\xe9\xa6\x96\xe9\xa1\xb5</a>\n <a href="http://weibo.com/" class="S_txt1">\xe5\xbe\xae\xe5\x8d\x9a</a>\n <a href="http://help.weibo.com/" class="S_txt1 last">\xe5\xb8\xae\xe5\x8a\xa9</a>\n </div>\n </div>\n <div class="iforgot_cont">\n <div class="i_mod">\n <div class="form_mod">\n <div class="form_list form_listError">\n <span class="iconError"></span>\n <span class="itemError code_mod">\xe7\xb3\xbb\xe7\xbb\x9f\xe6\x9c\x89\xe7\x82\xb9\xe5\xbf\x99\xef\xbc\x8c\xe8\xaf\xb7\xe5\x88\xb7\xe6\x96\xb0\xe4\xb8\x80\xe4\xb8\x8b\xe8\xaf\x95\xe8\xaf\x95</span>\n </div>\n </div>\n </div>\n </div>\n </div>\n</div>\n<div class="WB_footer S_bg2">\n <div class="other_link S_bg1 clearfix T_add_ser">\n <p class="copy"><a href="http://corp.sina.com.cn/chn/" class="footBg">\xe6\x96\xb0\xe6\xb5\xaa\xe7\xae\x80\xe4\xbb\x8b</a>\xe3\x80\x80<a class="footBg" href="http://corp.sina.com.cn/eng/">About Sina</a>\xe3\x80\x80<a class="footBg" href="http://emarketing.sina.com.cn/">\xe5\xb9\xbf\xe5\x91\x8a\xe6\x9c\x8d\xe5\x8a\xa1</a>\xe3\x80\x80<a class="footBg" href="http://www.sina.com.cn/contactus.html">\xe8\x81\x94\xe7\xb3\xbb\xe6\x88\x91\xe4\xbb\xac</a>\xe3\x80\x80<a class="footBg" href="http://corp.sina.com.cn/chn/sina_job.html">\xe6\x8b\x9b\xe8\x81\x98\xe4\xbf\xa1\xe6\x81\xaf</a>\xe3\x80\x80<a class="footBg" href="http://www.sina.com.cn/intro/lawfirm.shtml">\xe7\xbd\x91\xe7\xab\x99\xe5\xbe\x8b\xe5\xb8\x88</a>\xe3\x80\x80<a class="footBg" href="http://english.sina.com" target="__blank">SINA English</a>\xe3\x80\x80<a class="footBg" href="http://members.sina.com.cn/apply/" target="__blank">\xe6\xb3\xa8\xe5\x86\x8c</a>\xe3\x80\x80<a class="footBg" href="http://tech.sina.com.cn/focus/sinahelp.shtml" target="__blank">\xe4\xba\xa7\xe5\x93\x81\xe7\xad\x94\xe7\x96\x91</a></p>\n <div class="copy"><a href="javascript:;" class="S_txt2">\xe5\xae\xa2\xe6\x88\xb7\xe6\x9c\x8d\xe5\x8a\xa1\xe7\x94\xb5\xe8\xaf\x9d\xef\xbc\x9a400 052 0066 \xe6\xac\xa2\xe8\xbf\x8e\xe6\x89\xb9\xe8\xaf\x84\xe6\x8c\x87\xe6\xad\xa3</a></div>\n <p class="company"><span class="copy S_txt2">Copyright \xc2\xa9 1996-2020 SINA Corporation, All Rights Reserved \xe6\x96\xb0\xe6\xb5\xaa\xe5\x85\xac\xe5\x8f\xb8 \xe7\x89\x88\xe6\x9d\x83\xe6\x89\x80\xe6\x9c\x89</span></p>\n </div>\n </div>\n</body>\n</html>', 792, 793, 'illegal multibyte sequence') Traceback (most recent call last): File "/home/weibo/weibospider-temp_verification/.env/lib/python3.6/site-packages/celery/app/trace.py", line 375, in trace_task R = retval = fun(*args, **kwargs) File "/home/weibo/weibospider-temp_verification/.env/lib/python3.6/site-packages/celery/app/trace.py", line 632, in __protected_call__ return self.run(*args, **kwargs) File "/home/weibo/weibospider-temp_verification/tasks/login.py", line 12, in login_task get_session(name, password) File "/home/weibo/weibospider-temp_verification/login/login.py", line 230, in get_session url, cjy_client, cid, err_no, session = do_login(name, password, proxy) File "/home/weibo/weibospider-temp_verification/login/login.py", line 210, in do_login rs, cjy_client, cid, err_no, session = login_retry(name, password, session, cjy_client, cid, proxy, err_no) File "/home/weibo/weibospider-temp_verification/login/login.py", line 198, in login_retry proxy) File "/home/weibo/weibospider-temp_verification/login/login.py", line 184, in login_by_pincode rs = get_redirect(name, data, post_url, session, proxy) File "/home/weibo/weibospider-temp_verification/login/login.py", line 85, in get_redirect login_loop = logining_page.content.decode("gbk") UnicodeDecodeError: 'gbk' codec can't decode byte 0xae in position 792: illegal multibyte sequence [2020-06-28 23:25:45,264: ERROR/ForkPoolWorker-1] Task tasks.login.login_task[fa3de833-84ac-4f20-a81b-426cdfff7c97] raised unexpected: SyntaxError('invalid syntax', ('<string>', 1, 1, '<!doctype html>\n')) Traceback (most recent call last): File "/home/weibo/weibospider-temp_verification/.env/lib/python3.6/site-packages/celery/app/trace.py", line 375, in trace_task R = retval = fun(*args, **kwargs) File "/home/weibo/weibospider-temp_verification/.env/lib/python3.6/site-packages/celery/app/trace.py", line 632, in __protected_call__ return self.run(*args, **kwargs) File "/home/weibo/weibospider-temp_verification/tasks/login.py", line 12, in login_task get_session(name, password) File "/home/weibo/weibospider-temp_verification/login/login.py", line 230, in get_session url, cjy_client, cid, err_no, session = do_login(name, password, proxy) File "/home/weibo/weibospider-temp_verification/login/login.py", line 205, in do_login server_data = get_server_data(su, session, proxy) File "/home/weibo/weibospider-temp_verification/login/login.py", line 67, in get_server_data sever_data = eval(pre_data_res.content.decode("utf-8").replace("sinaSSOController.preloginCallBack", '')) File "<string>", line 1 <!doctype html> ^ SyntaxError: invalid syntax

4.你使用的是哪个版本的WeiboSpider? 你的操作系统是什么？是否有读本项目的常见问题？ 使用的是服务环境的ubuntu，环境配置已经弄过几次了

zjyzh commented 4 years ago

版本因为不兼容之前更换了：

kombu版本

pip install kombu==4.2.0

如果你也使用了celery

pip install celery==4.1.1

OneCodeMonkey commented 4 years ago

截图看一下 @zjyzh

OneCodeMonkey commented 4 years ago

云打码之前跑路，后来又恢复了一段时间，现在又跑路了。。

OneCodeMonkey commented 4 years ago

超级鹰我这几个月还没试过，之前用的挺好

zjyzh commented 4 years ago

image 这个是截图，想问下可以出一个稳定的超级鹰版本吗，非常感谢，因为项目的依赖关系不是轻易能弄懂的....

zjyzh commented 4 years ago

erro

zjyzh commented 4 years ago

我刚刚通过将login下面的代码改为text解决了上面的bug，但是它一直卡在登陆页面.... image

zjyzh commented 4 years ago

我在今天解决了这个问题，是因为超级鹰的余额不足以及账号不对引起的，希望大家注意下这些吧，麻烦了

OneCodeMonkey commented 4 years ago

好的