报错 - Githubissues

okoling commented 2 years ago

config.yaml

root@VM-4-16-ubuntu:~/Sec# cat config.yml
ChromeDriver: /usr/bin/google-chrome
Proxy:
  ProxyUrl: http://127.0.0.1:7890
  CrawlerProxyEnabled: false
  BotProxyEnabled: false

报错

[!] 2022/02/25 02:22:05 crawl [XianZhi] error: server did not respond on port 29515

2022/02/24 07:45:00
推荐工程系统架构演进
https://mp.weixin.qq.com/s?__biz=MzA5OTAyNzQ2OA==&amp;mid=2649753510&amp;idx=1&amp;sn=4252de19327168fbf6a1721f3389ea9e

2022/02/24 07:42:05
洞见简报【2022/2/23】
https://mp.weixin.qq.com/s?__biz=MzAxNzg3NzMyNQ==&amp;mid=2247485643&amp;idx=1&amp;sn=7ad37761db712112741d4b9051ed1061

......
......
......
[*] send to DingBot: {"errcode":460101,"errmsg":"message too long, exceed 20000 bytes"}

Le0nsec commented 2 years ago

ChromeDriver这里应该填的是ChromeDriver的地址，不是chrome的地址，所以先知社区爬不了，没有正确调用。
洞见微信聚合这个一次获取的内容过多，导致超出了机器人接收长度，会尽快修改。

okoling commented 2 years ago

解决了，安装完google-chrome-stable_current_amd64.deb下载对应ChromeDriver就行了。
洞见这类链接https://mp.weixin.qq.com/s?__biz=MzAxNzg3NzMyNQ==&mid=2247485643&idx=1&sn=7ad37761db712112741d4b9051ed1061占用字节过多，压缩成短链接发送？

Le0nsec commented 2 years ago

已经把爬取的url改为了精华分类的并且暂时限制为10篇，等找到合适且稳定的短链接服务再优化

Le0nsec / SecCrawler

报错 #10