Open WarlockFish opened 6 years ago
4月14日更新:用openshift部署了一下,一下就部署成功。因为没有https加密,部署后的网站很容易被墙,但该网站不会休眠
5月2日更新:
目前这个网站确实上不去了,https://gentle-oasis-59484.herokuapp.com 是不是和运营商有关系呢?昨天晚上还是正常的
之前可以直接上 ,现在要翻墙
不太清楚具体什么情况,我在国内不开代理也能上的了的,不过既然本项目开源,大家也可以自行部署:)
目测苏移动可访
目前该网站仍需要翻墙才能访问,本人教育网
目前这个网站在我这边确实需要翻墙才能访问
同需要翻墙才能访问
既然有不少朋友都遇到无法访问的问题,建议朋友们按照本项目readme的指南自己注册个heroku账号并搭建本网站,并在本issue下面回复一下你自己搭建的网站地址:)
部署不需要写代码/装软件,完全免费,既方便又简单,应该也不用担心被屏蔽:)
我不换域名/IP的原因是:
https://sharefanqiang.herokuapp.com/ 这是我刚刚搭建的网站地址
该网站的订阅地址是什么?
网站免费小时数已耗尽,得等到下个月才能重新开启了:)
网站更新周期变量从哪里添加?https://zheng123456.herokuapp.com 感觉现在进去几次就重新抓取了,频率太快。
最新版本在heroku中添加一个环境变量,PERIOD,里面的数值代表访问网站的次数,包括更新订阅等任何访问均会增加访问计数
period变量不起作用。网页上的地址刷新几次就变没了。 好像不是根据变量刷新,是根据时间。period没用呢。
地址都没了是因为heroku检测到长时间没人使用这个网站,就把server关掉了
都是全局模式,通过ssr,谷歌可以登录,但是就是无法更新订阅 怎么回事啊???
那能不让heroku关掉server吗? 好像非要买Dynos才不会进入sleep.
@from021 订阅更新不了是因为服务器本月配额用完了 @littleguyy 要是验证信用卡还可以增加server的时间
我测试了一下,不在浏览器里打开heroku的应用,只通过ssr更新应用,应用后台显示出http的get网页,我换了一个ip继续更新,后台也有不同的ip发来get请求,但是应用就是不开始爬,我用浏览器打开应用,应用的访问计数还是1,我用网页刷新了一下才变成2开始爬。 说明ssr更新,不能让应用计数。不能让它爬。
能不能把程序做成本地电脑运行呢?
最新版已经可以实现让ssr订阅也增加访问计数了,也就是说订阅更新也会触发爬虫;
可以在电脑里做到本地运行,具体的你可以将本项目代码下载到本地,使用
from app.ss import ss_free
servers = ss_free.main()
返回的这个servers里面包含了你所需要的全部信息,不过爬虫比较费时间。
不过不建议这样做,因为我爬的网站几乎都是大陆无法访问的,如果你在大陆使用这种方法通常只能爬到很少的结果
好的,谢谢了。
HELP!我按照教程申请了账号,fork了项目,添加了程序,增加了变量,运行之后啥都抓不到,始终显示“服务器通知:别着急,正在爬数据,十分钟后再回来吧:) ”,请教该怎么办? https://mw-ssr.herokuapp.com/
把日志贴一下吧? 在heroku控制面板右上角的more,有个view logs
2017-12-12T08:47:43.836718+00:00 app[web.1]: 2017-12-12 08:47:43 ERROR tcprelay.py:1039 [Errno 111] Connection refused 2017-12-12T08:47:43.836850+00:00 app[web.1]: 2017-12-12 08:47:43 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T08:47:43.838183+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T08:47:40.511764+00:00 app[web.1]: >>>>>>>>>> 结果: True 2017-12-12T08:47:40.511823+00:00 app[web.1]: >>>>>>>>>> 测试通过! 2017-12-12T08:47:40.511875+00:00 app[web.1]: {'message': '', 'name': 'NoBey', 'url': 'https://github.com/NoBey/Shadowsocks-free'} 2017-12-12T08:47:46.841078+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T08:47:49.079146+00:00 app[web.1]: 10.61.234.242 - - [12/Dec/2017 08:47:49] "GET / HTTP/1.1" 200 - 2017-12-12T08:47:49.079408+00:00 app[web.1]: 2017-12-12 08:47:49 INFO _internal.py:87 10.61.234.242 - - [12/Dec/2017 08:47:49] "GET / HTTP/1.1" 200 - 2017-12-12T08:47:49.971182+00:00 app[web.1]: 2017-12-12 08:47:49 ERROR tcprelay.py:1039 [Errno 111] Connection refused 2017-12-12T08:47:49.971374+00:00 app[web.1]: 2017-12-12 08:47:49 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T08:47:49.972802+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T08:47:49.986107+00:00 app[web.1]: 10.61.234.242 - - [12/Dec/2017 08:47:49] "GET /favicon.ico HTTP/1.1" 200 - 2017-12-12T08:47:50.010400+00:00 app[web.1]: 2017-12-12 08:47:49 INFO _internal.py:87 10.61.234.242 - - [12/Dec/2017 08:47:49] "GET /favicon.ico HTTP/1.1" 200 - 2017-12-12T08:47:50.018164+00:00 heroku[router]: at=info method=GET path="/favicon.ico" host=mw-ssr.herokuapp.com request_id=faa67a08-e867-4a33-91a2-f1ff62774405 fwd="124.160.63.37" dyno=web.1 connect=0ms service=45ms status=200 bytes=166004 protocol=https 2017-12-12T08:47:51.018120+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T08:47:49.081182+00:00 heroku[router]: at=info method=GET path="/" host=mw-ssr.herokuapp.com request_id=6bff0bc7-90e4-4e5d-8e22-2fe57625ae76 fwd="124.160.63.37" dyno=web.1 connect=0ms service=11ms status=200 bytes=7693 protocol=https 2017-12-12T08:47:54.103198+00:00 app[web.1]: 2017-12-12 08:47:54 ERROR tcprelay.py:1039 [Errno 111] Connection refused 2017-12-12T08:47:54.103390+00:00 app[web.1]: 2017-12-12 08:47:54 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T08:47:54.104163+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T08:47:55.105027+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T08:47:58.191741+00:00 app[web.1]: 2017-12-12 08:47:58 ERROR tcprelay.py:1039 [Errno 111] Connection refused 2017-12-12T08:47:58.191822+00:00 app[web.1]: 2017-12-12 08:47:58 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T08:47:58.192372+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T08:48:01.193816+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T08:48:04.276371+00:00 app[web.1]: 2017-12-12 08:48:04 ERROR tcprelay.py:1039 [Errno 111] Connection refused 2017-12-12T08:48:04.276474+00:00 app[web.1]: 2017-12-12 08:48:04 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T08:48:04.277067+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T08:48:07.279159+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T08:48:10.648820+00:00 app[web.1]: 2017-12-12 08:48:10 ERROR tcprelay.py:1039 [Errno 111] Connection refused 2017-12-12T08:48:10.648958+00:00 app[web.1]: 2017-12-12 08:48:10 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T08:48:10.649773+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T08:48:13.652022+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T08:48:16.741675+00:00 app[web.1]: 2017-12-12 08:48:16 ERROR tcprelay.py:1039 [Errno 111] Connection refused 2017-12-12T08:48:16.741777+00:00 app[web.1]: 2017-12-12 08:48:16 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T08:48:16.742641+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T08:48:22.811879+00:00 app[web.1]: 2017-12-12 08:48:22 ERROR tcprelay.py:1039 [Errno 111] Connection refused 2017-12-12T08:48:22.811950+00:00 app[web.1]: 2017-12-12 08:48:22 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T08:48:22.812715+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T08:48:19.744305+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T08:48:25.814581+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T08:48:28.881149+00:00 app[web.1]: 2017-12-12 08:48:28 ERROR tcprelay.py:1039 [Errno 111] Connection refused 2017-12-12T08:48:28.881226+00:00 app[web.1]: 2017-12-12 08:48:28 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T08:48:28.881756+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T08:48:31.884537+00:00 app[web.1]: >>>>>>>>>> 结果: False
一直在重复……
Disconnected from log stream. There may be events happening that you do not see here! Attempting to reconnect... 2017-12-12T17:00:33.297348+00:00 app[web.1]: SOCKSHTTPConnectionPool(host='ip.cn', port=80): Read timed out. (read timeout=10) 2017-12-12T17:00:34.298347+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:37.466073+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T17:00:37.465287+00:00 app[web.1]: 2017-12-12 17:00:37 ERROR tcprelay.py:1039 [Errno 111] Connection refused 2017-12-12T17:00:37.465403+00:00 app[web.1]: 2017-12-12 17:00:37 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T17:00:40.467725+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:53.484641+00:00 app[web.1]: SOCKSHTTPConnectionPool(host='ip.cn', port=80): Read timed out. (read timeout=10) 2017-12-12T17:00:54.486259+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:57.690196+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T17:01:00.692385+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:01:00.692460+00:00 app[web.1]: ---------- 数据获取完毕 ---------- 2017-12-12T17:16:07.883249+00:00 heroku[web.1]: Idling 2017-12-12T17:16:07.883888+00:00 heroku[web.1]: State changed from up to down 2017-12-12T17:16:08.908950+00:00 heroku[web.1]: Stopping all processes with SIGTERM 2017-12-12T17:16:09.108371+00:00 heroku[web.1]: Process exited with status 143 Disconnected from log stream. There may be events happening that you do not see here! Attempting to reconnect... 2017-12-12T17:00:37.465403+00:00 app[web.1]: 2017-12-12 17:00:37 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T17:00:40.467725+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:53.484641+00:00 app[web.1]: SOCKSHTTPConnectionPool(host='ip.cn', port=80): Read timed out. (read timeout=10) 2017-12-12T17:00:54.486259+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:57.690196+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T17:01:00.692385+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:01:00.692460+00:00 app[web.1]: ---------- 数据获取完毕 ---------- 2017-12-12T17:16:07.883249+00:00 heroku[web.1]: Idling 2017-12-12T17:16:07.883888+00:00 heroku[web.1]: State changed from up to down 2017-12-12T17:16:08.908950+00:00 heroku[web.1]: Stopping all processes with SIGTERM 2017-12-12T17:16:09.108371+00:00 heroku[web.1]: Process exited with status 143 Disconnected from log stream. There may be events happening that you do not see here! Attempting to reconnect... 2017-12-12T17:00:37.465403+00:00 app[web.1]: 2017-12-12 17:00:37 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T17:00:40.467725+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:53.484641+00:00 app[web.1]: SOCKSHTTPConnectionPool(host='ip.cn', port=80): Read timed out. (read timeout=10) 2017-12-12T17:00:54.486259+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:57.690196+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T17:01:00.692385+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:01:00.692460+00:00 app[web.1]: ---------- 数据获取完毕 ---------- 2017-12-12T17:16:07.883249+00:00 heroku[web.1]: Idling 2017-12-12T17:16:07.883888+00:00 heroku[web.1]: State changed from up to down 2017-12-12T17:16:08.908950+00:00 heroku[web.1]: Stopping all processes with SIGTERM 2017-12-12T17:16:09.108371+00:00 heroku[web.1]: Process exited with status 143
https://wyy.herokuapp.com/ 我也建成功啦,哈哈
https://eyyy.herokuapp.com/ 建了一个,抓不到啊
主网站已被墙,大家可以自己搭
你的意思是搭建成功后的一段时间出现Application error是到期了,对吧
对,每月月底就没有流量了
@the0demiurge 网站又上不去了
订阅地址能不能支持下https mac客户端只能用https。感谢!!!
@ruiyanan 你可以用 https://aoxuaudaadaoeu.herokuapp.com ,不过每个月月底会没流量
https://wyy.herokuapp.com/ 我也建成功啦,哈哈
认证了信用卡,现在每月有1G流量
这里也有免费账号https://www.myexplor.me/33ssray.html
http屏蔽是当然的;https屏蔽应该是SNI阻断,可以在对应平台(如heroku)找一个正常且不被屏蔽的项目,通过SNI欺骗+nginx本地反向代理的方式访问 具体而言,
Browser [notallowed.example.com] ---> Nginx Server (in China)[change SNI to allowed.example.com] ---> GFW [SNI checked,passed] ---> Target (out of China)[notallowed.example.com]
前提是notallowed.example.com不在意SNI与Host不匹配且能提供正确SSL证书,且IP未被屏蔽 类似于meek工作原理,不同的是不经过公有云 部分抄袭Wikipedia解决方案
因为heroku说我的网站违反了政策,网站被暂停了,现在正在申诉中
感谢大佬,成功建站,并且已经爬到了! 不过我怎么更改爬虫的频率呢?
大多数时间显示的是这个提示!!!
有的时候长时间没人访问网站heroku就把网站suspend了,然后你访问网站之前的数据就没了,会重新爬数据,会显示这个
现在爬的账号比较多,应该要等不止十分钟。访问量大的网站出现这种情况的频率会比较少,或者如果自己用VPS搭建网站的话不会出现这种情况
On Tue, Nov 12, 2019, 10:42 AM linuxhobby notifications@github.com wrote:
别着急,正在爬数据,十分钟后再回来吧:)
大多数时间显示的是这个提示!!!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/the0demiurge/ShadowSocksShare/issues/17?email_source=notifications&email_token=ACWWJSNPRVZDYTCMBM5PSVLQTIJXZA5CNFSM4EELPN2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDY2UAA#issuecomment-552708608, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWWJSNIE7UL34KIW5MVCALQTIJXZANCNFSM4EELPN2A .
Disconnected from log stream. There may be events happening that you do not see here! Attempting to reconnect... 2017-12-12T17:00:33.297348+00:00 app[web.1]: SOCKSHTTPConnectionPool(host='ip.cn', port=80): Read timed out. (read timeout=10) 2017-12-12T17:00:34.298347+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:37.466073+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T17:00:37.465287+00:00 app[web.1]: 2017-12-12 17:00:37 ERROR tcprelay.py:1039 [Errno 111] Connection refused 2017-12-12T17:00:37.465403+00:00 app[web.1]: 2017-12-12 17:00:37 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T17:00:40.467725+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:53.484641+00:00 app[web.1]: SOCKSHTTPConnectionPool(host='ip.cn', port=80): Read timed out. (read timeout=10) 2017-12-12T17:00:54.486259+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:57.690196+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T17:01:00.692385+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:01:00.692460+00:00 app[web.1]: ---------- 数据获取完毕 ---------- 2017-12-12T17:16:07.883249+00:00 heroku[web.1]: Idling 2017-12-12T17:16:07.883888+00:00 heroku[web.1]: State changed from up to down 2017-12-12T17:16:08.908950+00:00 heroku[web.1]: Stopping all processes with SIGTERM 2017-12-12T17:16:09.108371+00:00 heroku[web.1]: Process exited with status 143 Disconnected from log stream. There may be events happening that you do not see here! Attempting to reconnect... 2017-12-12T17:00:37.465403+00:00 app[web.1]: 2017-12-12 17:00:37 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T17:00:40.467725+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:53.484641+00:00 app[web.1]: SOCKSHTTPConnectionPool(host='ip.cn', port=80): Read timed out. (read timeout=10) 2017-12-12T17:00:54.486259+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:57.690196+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T17:01:00.692385+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:01:00.692460+00:00 app[web.1]: ---------- 数据获取完毕 ---------- 2017-12-12T17:16:07.883249+00:00 heroku[web.1]: Idling 2017-12-12T17:16:07.883888+00:00 heroku[web.1]: State changed from up to down 2017-12-12T17:16:08.908950+00:00 heroku[web.1]: Stopping all processes with SIGTERM 2017-12-12T17:16:09.108371+00:00 heroku[web.1]: Process exited with status 143 Disconnected from log stream. There may be events happening that you do not see here! Attempting to reconnect... 2017-12-12T17:00:37.465403+00:00 app[web.1]: 2017-12-12 17:00:37 ERROR tcprelay.py:1041 remote error, when connect to 23.239.1.72:80 2017-12-12T17:00:40.467725+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:53.484641+00:00 app[web.1]: SOCKSHTTPConnectionPool(host='ip.cn', port=80): Read timed out. (read timeout=10) 2017-12-12T17:00:54.486259+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:00:57.690196+00:00 app[web.1]: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) 2017-12-12T17:01:00.692385+00:00 app[web.1]: >>>>>>>>>> 结果: False 2017-12-12T17:01:00.692460+00:00 app[web.1]: ---------- 数据获取完毕 ---------- 2017-12-12T17:16:07.883249+00:00 heroku[web.1]: Idling 2017-12-12T17:16:07.883888+00:00 heroku[web.1]: State changed from up to down 2017-12-12T17:16:08.908950+00:00 heroku[web.1]: Stopping all processes with SIGTERM 2017-12-12T17:16:09.108371+00:00 heroku[web.1]: Process exited with status 143
解决了吗
https://ss-collection.herokuapp.com/
我部署的,还在等待看是否能爬到
佛系,躺着,南无阿弥陀佛
今天上网时,发现https://gentle-oasis-59484.herokuapp.com 上不去