my8100 / scrapyd-cluster-on-heroku

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO :point_right:
https://scrapydweb.herokuapp.com/
GNU General Public License v3.0
122 stars 94 forks source link

Fail to deploy the scrapydweb app on Heroku #1

Closed leffss closed 5 years ago

leffss commented 5 years ago

完全照你的教程,scrapydweb无法发布到heroku呢

my8100 commented 5 years ago

Any log or screenshots?

leffss commented 5 years ago
  1. scrapyd 正常

2.scrapydweb 不正常

  1. scrapydweb heroku日志见附件

  2. scrapydweb heroku配置见附件

注: scrapyd是用你项目搭建的,没改任何配置;scrapydweb也是,只改了scrapydweb_settings_v8.py配置。

------------------ 原始邮件 ------------------ 发件人: "LxL"notifications@github.com; 发送时间: 2019年4月18日(星期四) 晚上6:43 收件人: "my8100/scrapyd-cluster-on-heroku"scrapyd-cluster-on-heroku@noreply.github.com; 抄送: "李发富"348926676@qq.com; "Author"author@noreply.github.com; 主题: Re: [my8100/scrapyd-cluster-on-heroku] scrapydweb无法发布到heroku (#1)

Any log or screenshots?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

leffss commented 5 years ago

Any log or screenshots? 日志和截图也发了你的gmail一份

my8100 commented 5 years ago

Key log

2019-04-19T00:42:41.163563+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch

Faulty config

# Accept connections on the specified port, the default is 5000.
SCRAPYDWEB_PORT = 5000
# SCRAPYDWEB_PORT = int(os.environ.get('PORT', 5000))

Just uncomment the SCRAPYDWEB_PORT = int(os.environ.get('PORT', 5000)) to address your problem. (See https://github.com/my8100/scrapyd-cluster-on-heroku-scrapyd-app/issues/1#issuecomment-481580396)

It's recommended to Deploy Heroku apps in the browser and set up necessary config options via environment variables in the web UI of Heroku.

leffss commented 5 years ago

Key log

2019-04-19T00:42:41.163563+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch

Faulty config

# Accept connections on the specified port, the default is 5000.
SCRAPYDWEB_PORT = 5000
# SCRAPYDWEB_PORT = int(os.environ.get('PORT', 5000))

Just uncomment the SCRAPYDWEB_PORT = int(os.environ.get('PORT', 5000)) to address your problem. (See my8100/scrapyd-cluster-on-heroku-scrapyd-app#1 (comment))

It's recommended to Deploy Heroku apps in the browser and set up necessary config options via environment variables in the web UI of Heroku.

thanks! 解决了。但是有个疑问: scrapyd的heroku运行文件中的 init.py 中第17行: args.extend(['--scrapyd_server', os.environ.get('SCRAPYD_SERVER', '127.0.0.1:6800')]) 我觉得有点问题呢,不该是这样合理一点吗? args.extend(['--scrapyd_server', os.environ.get('SCRAPYD_SERVER', '127.0.0.1:{0}'.format(PORT))])

my8100 commented 5 years ago

Thanks for your suggestion. Now the actual PORT would be shown in the log, but it's not exposed externally, which means you still can't make use of it.

2019-04-19T04:43:21.023281+00:00 app[web.1]: [2019-04-19 04:43:21,023] WARNING  in logparser.logparser: Saved to http://127.0.0.1:29760/logs/stats.json