search

howie6879 / liuli

一站式构建多源、干净、个性化的阅读环境(Build a multi-source, clean and personalized reading environment in one stop.)
https://liuli.io
Apache License 2.0
886 stars 109 forks source link

求助：日志正常，但备份器内容为空 #82

Closed SkeStars closed 1 year ago

SkeStars commented 1 year ago

Docker Compose部署，日志感觉正常，但备份器内容为空，求大佬看看哪里有问题

RSS： image

github备份：https://SkeStars.github.io/liuli_backup/liuli_wechat/老胡的储物柜/老胡的周刊（第101期）.html
404 没有数据

mongodb备份：http://192.168.1.10:8765/rss/liuli_wechat/老胡的储物柜/老胡的周刊（第101期） 页面加载完成是一片空白

pro.env:

# ======================================系统环境配置======================================#
# 当前目录为模块
PYTHONPATH=${PYTHONPATH}:${PWD}

# =======================================数据库配置=======================================#
# MongoDB 用户名
LL_M_USER="liuli"
# MongoDB 密码
LL_M_PASS="liuli"
# MongoDB IP
# Docker Compose 形式启动的话，此行配置不变
LL_M_HOST="liuli_mongodb"
# MongoDB 端口
LL_M_PORT="27017"
# MongoDB 默认 db
# 如果用户自建db，填写自己的即可 此时 LL_M_OP_DB 可不填
LL_M_DB="admin"
LL_M_OP_DB="liuli"

# ======================================接口服务配置======================================#
# Flask 是否开启Flask的Debug模式
LL_HTTP_DEBUG=0
# Flask IP
LL_HTTP_HOST="0.0.0.0"
# Flask 端口
LL_HTTP_PORT=8765
# 访问域名，没有域名填本机实际地址（因为要开放对外访问），如: http://192.168.0.1:8765
LL_DOMAIN="http://192.168.1.10:8765"
# Flask 服务启动的 worker 数量
LL_HTTP_WORKERS=1

# =======================================分发器配置=======================================#
# 分发器终端配置，用户在环境变量配置好密钥后，在启动配置的 sender.sender_list 填写好想分发的终端即可
# 目前支持：ding[钉钉] wecom[企业微信] tg[Telegram] Bark
# 分发终端为钉钉必须配置的Token
LL_DD_TOKEN=""
# 分发终端为企业微信的配置，如果不配置分发用户与部门，则默认会发送给所有部门的所有用户
LL_WECOM_ID=""
LL_WECOM_AGENT_ID="-1"
LL_WECOM_SECRET=""
# 企业微信分发用户（填写用户帐号，不区分大小写），多个用户用;分割
LL_WECOM_TO_USER=""
# 企业微信分发部门（填写部门名称），多个部门用;分割
LL_WECOM_PARTY=""
# TG 终端配置
LL_TG_CHAT_ID=""
LL_TG_TOKEN=""
# Bark推送链接
LL_BARK_URL=""

# =======================================备份器配置=======================================#
# 备份器目前支持: github mongodb
# 使用 mongodb 备份的话则默认使用上面配置的数据库地址进行备份
# 使用 github 备份的话需要填写以下配置
# 项目权限token
LL_GITHUB_TOKEN="ghp_......"
# 文章保存项目地址，例：howie6879/liuli_backup 项目名称一定为 liuli_backup
LL_GITHUB_REPO="SkeStars/liuli_backup"
# 访问域名，可自定义也可以用默认的，如果用github做备份器就必填，以我个人备份项目为例地址为：https://howie6879.github.io/liuli_backup/
LL_GITHUB_DOMAIN="https://SkeStars.github.io/liuli_backup"
~

wechat.json：

{
  "name": "wechat",
  "username": "liuli",
  "author": "liuli_team",
  "doc_source": "liuli_wechat_sg",
  "doc_source_alias_name": "微信源(搜狗)",
  "collector": {
    "wechat": {
      "wechat_list": ["老胡的储物柜", "是不是很酷"],
      "delta_time": 5,
      "spider_type": "sg_ruia",
      "spider_type_des": "当镜像是schedule:playwright_*时，spider_type可填写sg_playwright"
    }
  },
  "processor": {
    "before_collect": [],
    "after_collect": [
      {
        "func": "ad_marker",
        "cos_value": 0.6
      },
      {
        "func": "to_rss",
        "doc_source_list": ["liuli_wechat"],
        "link_source": "github"
      }
    ]
  },
  "sender": {
    "sender_list": ["wecom"],
    "query_days": 7,
    "delta_time": 3,
    "custom_filter": {
      "wecom": {
        "delta_time": 1,
        "ignore_doc_source_name": [""]
      }
    }
  },
  "backup": {
    "backup_list": ["github", "mongodb"],
    "query_days": 7,
    "delta_time": 3,
    "init_config": {},
    "after_get_content": [
      {
        "func": "str_replace",
        "before_str": "data-src=\"",
        "after_str": "src=\"https://images.weserv.nl/?url="
      }
    ]
  },
  "schedule": {
    "period_list": ["00:10", "12:10", "21:10"]
  }
}

日志：

liuli_schedule
date,stream,content
2023/07/31 11:48:04,stderr,[2023:07:31 11:48:04] INFO  Liuli 备份器执行完毕!

2023/07/31 11:48:04,stderr,[2023:07:31 11:48:04] INFO  Liuli 备份器开始执行!

2023/07/31 11:48:04,stderr,[2023:07:31 11:48:04] INFO  Liuli 分发器执行完毕!

2023/07/31 11:48:04,stderr,[2023:07:31 11:48:04] INFO  Liuli 分发器开始执行!

2023/07/31 11:48:04,stderr,[2023:07:31 11:48:04] INFO  Liuli 处理器(after_collect): 执行完毕!

2023/07/31 11:48:04,stderr,[2023:07:31 11:48:04] INFO  Liuli 😀 为liuli_wechat: 是不是很酷 的 1 篇文章生成RSS成功!

2023/07/31 11:48:04,stderr,[2023:07:31 11:48:04] INFO  Liuli 😀 为liuli_wechat: 老胡的储物柜 的 1 篇文章生成RSS成功!

2023/07/31 11:48:02,stderr,[2023:07:31 11:48:02] INFO  Liuli 处理器(after_collect): to_rss 正在执行...

2023/07/31 11:48:01,stderr,[2023:07:31 11:48:01] INFO  Liuli 处理器(after_collect): ad_marker 正在执行...

2023/07/31 11:48:01,stderr,[2023:07:31 11:48:01] INFO  Liuli 处理器(after_collect): 开始执行!

2023/07/31 11:48:01,stderr,[2023:07:31 11:48:01] INFO  Liuli 采集器执行完毕!

2023/07/31 11:48:01,stderr,[2023:07:31 11:48:01] INFO  Liuli 🤗 微信公众号文章更新完毕(2/2)!

2023/07/31 11:48:01,stderr,[2023:07:31 11:48:01] INFO  SGWechatSpider Spider finished!

2023/07/31 11:48:01,stderr,[2023:07:31 11:48:01] INFO  SGWechatSpider Time usage: 0:00:24.225777

2023/07/31 11:48:01,stderr,[2023:07:31 11:48:01] INFO  SGWechatSpider Total requests: 3

2023/07/31 11:48:01,stderr,[2023:07:31 11:48:01] INFO  SGWechatSpider Stopping spider: SGWechatSpider

2023/07/31 11:48:01,stderr,[2023:07:31 11:48:01] INFO  Liuli 来自 liuli_wechat 的文章持久化成功! 👉 是不是很酷: 塞班程序员，没有一夜失业 

2023/07/31 11:47:53,stderr,[2023:07:31 11:47:52] INFO  Request <GET: http://mp.weixin.qq.com/s?src=11&timestamp=1690775262&ver=4683&signature=mhbbuTtWFTI7CdWs8rxERbl71rc-mIXcsFo5cJjWQTCMIN4iA94rA2QPeJOui*CnOHpWzQDB7*3lgpbPfgVkK8bJap5IrcDCz*EkQEnBCqLvQRHjgiBbs8cQizt9n8Pf&new=1>

2023/07/31 11:47:47,stderr,[2023:07:31 11:47:47] INFO  Request <GET: https://weixin.sogou.com//link?url=dn9a_-gY295K0Rci_xozVXfdMkSQTLW6cwJThYulHEtVjXrGTiVgSzrFFbu7iCCaXJCmTZojYuOoV5bKBOCl6FqXa8Fplpd9UYMz50qxDrMbWQp702idZwR1fgsNV1pSkOyXVNvNEjQft_rno_r8zhLGii0Es1BKfkZnl3Z3ae8uB45uGJAX4WtDvwkfRp-AiBpb9tfryGwiRPdVogG0X0Y9d7OmTRBTPtwqslUp3tMTaLtHjC38Gs7z6LC_d6amEtzL1uDyNvhj32-j2KiwUw..&type=1&query=%E6%98%AF%E4%B8%8D%E6%98%AF%E5%BE%88%E9%85%B7&token=7D56073FD7564CB8C9CFCB0A80201FF1CAA8638164C72EDE>

2023/07/31 11:47:42,stderr,[2023:07:31 11:47:42] INFO  Request <GET: https://weixin.sogou.com/weixin?type=1&query=是不是很酷&ie=utf8&s_from=input&_sug_=n&_sug_type_=>

2023/07/31 11:47:37,stderr,[2023:07:31 11:47:37] INFO  SGWechatSpider Worker started: 547483222288

2023/07/31 11:47:37,stderr,[2023:07:31 11:47:37] INFO  SGWechatSpider Worker started: 547488572704

2023/07/31 11:47:37,stderr,[2023:07:31 11:47:37] INFO  SGWechatSpider Spider started!

2023/07/31 11:47:37,stderr,[2023:07:31 11:47:37] INFO  SGWechatSpider Spider finished!

2023/07/31 11:47:37,stderr,[2023:07:31 11:47:37] INFO  SGWechatSpider Time usage: 0:00:41.143046

2023/07/31 11:47:37,stderr,[2023:07:31 11:47:37] INFO  SGWechatSpider Total requests: 3

2023/07/31 11:47:37,stderr,[2023:07:31 11:47:37] INFO  SGWechatSpider Stopping spider: SGWechatSpider

2023/07/31 11:47:37,stderr,[2023:07:31 11:47:37] INFO  Liuli 来自 liuli_wechat 的文章持久化成功! 👉 老胡的储物柜: 老胡的周刊（第101期） 

2023/07/31 11:47:27,stderr,[2023:07:31 11:47:27] INFO  Request <GET: http://mp.weixin.qq.com/s?src=11&timestamp=1690775224&ver=4683&signature=TiU4oox3Q4Z9C7uQTCcich-6Ag-f*vXw-TZas*2L7bGsx8fbY5Rx1Su8AE6KLiUguILdQMmyLOu2FvxWBtUQVKHorkWxKxh5A4gl*NkTCNtIFki-CpyNEJCHHPk7pXQ2&new=1>

2023/07/31 11:47:19,stderr,[2023:07:31 11:47:11] INFO  Request <GET: https://weixin.sogou.com//link?url=dn9a_-gY295K0Rci_xozVXfdMkSQTLW6cwJThYulHEtVjXrGTiVgSzrFFbu7iCCarJVdx82-JxyoV5bKBOCl6FqXa8Fplpd9g1VC5TPn4xZsMRcHtdonQs6b-WS8f9BsO_heBtNWVrUVeENb58czLtDFx1VphRjlrpZDT7qzPE6OyAebdcNB9bdTnJU1PpFfH-nlQZe4U-4WlA7SCr1AKBYH9hGZS8gTc-uFAiF2oLlSadiezxd9ZjQI6MyyNfFfqKkX31A-ufGTJmCU1UgHwQ..&type=1&query=%E8%80%81%E8%83%A1%E7%9A%84%E5%82%A8%E7%89%A9%E6%9F%9C&token=7D5567188A0815E594929253DC0B410C94F5DCEF64C72EB8>

2023/07/31 11:47:01,stderr,[2023:07:31 11:47:01] INFO  Request <GET: https://weixin.sogou.com/weixin?type=1&query=老胡的储物柜&ie=utf8&s_from=input&_sug_=n&_sug_type_=>

2023/07/31 11:46:56,stderr,[2023:07:31 11:46:56] INFO  SGWechatSpider Worker started: 547483222288

2023/07/31 11:46:56,stderr,[2023:07:31 11:46:56] INFO  SGWechatSpider Worker started: 547488572704

2023/07/31 11:46:56,stderr,[2023:07:31 11:46:56] INFO  SGWechatSpider Spider started!

2023/07/31 11:46:46,stderr,[2023:07:31 11:46:46] INFO  Liuli 采集器开始执行!

2023/07/31 11:46:46,stderr, 21:10

2023/07/31 11:46:46,stderr, 12:10

2023/07/31 11:46:46,stderr, 00:10

2023/07/31 11:46:46,stderr,[2023:07:31 11:46:46] INFO  Liuli Task(wechat@liuli_team) schedule time:

2023/07/31 11:46:46,stderr,[2023:07:31 11:46:46] INFO  Liuli Schedule(v0.2.4) task(wechat@liuli_team) started successfully :)

2023/07/31 11:46:12,stderr,Loading .env environment variables...

2023/07/31 11:46:00,stdout,Start schedule(pro) serve: PIPENV_DOTENV_LOCATION=./pro.env pipenv run python src/liuli_schedule.py

2023/07/31 11:45:58,stderr,Loading .env environment variables...
SkeStars commented 1 year ago

wechat.json： "doc_source": "liuli_wechat_sg" -> "liuli_wechat"