LetTTGACO / elog

Markdown 批量导出工具、开放式跨平台博客解决方案,随意组合写作平台(语雀/Notion/FlowUs/飞书/我来Wolai)和博客平台(Hexo/Vitepress/Halo/Confluence/WordPress等)
https://elog.1874.cool
MIT License
1.2k stars 58 forks source link

飞书图片下载 api 请求过于频繁导致的错误 #90

Closed xiehuacheng closed 1 day ago

xiehuacheng commented 6 days ago

你当前使用的Elog版本

0.14.2-beta.0

Elog配置文件

module.exports = {
  write: {
    platform: 'feishu',
    yuque: {
      token: process.env.YUQUE_TOKEN,
      login: process.env.YUQUE_LOGIN,
      repo: process.env.YUQUE_REPO,
      onlyPublic: false,
      onlyPublished: true,
    },
    'yuque-pwd': {
      username: process.env.YUQUE_USERNAME,
      password: process.env.YUQUE_PASSWORD,
      login: process.env.YUQUE_LOGIN,
      repo: process.env.YUQUE_REPO,
      onlyPublic: false,
      onlyPublished: true,
    },
    notion: {
      token: process.env.NOTION_TOKEN,
      databaseId: process.env.NOTION_DATABASE_ID,
      filter: false, // {property: 'status', select: {equals: '已发布'}}
    },
    feishu: {
      type: 'space',
      wikiId: process.env.FEISHU_WIKI_ID,
      folderToken: process.env.FEISHU_FOLDER_TOKEN,
      appId: process.env.FEISHU_APP_ID,
      appSecret: process.env.FEISHU_APP_SECRET,
    },
    flowus: {
      tablePageId: process.env.FLOWUS_TABLE_PAGE_ID,
      filter: false, // {property: 'status',value: '已发布'}
    },
    wolai: {
      token: process.env.WOLAI_TOKEN,
      pageId: process.env.WOLAI_PAGE_ID,
    }
  },
  deploy: {
    platform: 'wordpress',
    local: {
      outputDir: './docs',
      filename: 'title',
      format: 'markdown',
    },
    halo: {
      endpoint: process.env.HALO_ENDPOINT,
      token: process.env.HALO_TOKEN,
      policyName: process.env.HALO_POLICY_NAME,
      rowType: 'html',
      needUploadImage: true,
    },
    confluence: {
      user: process.env.CONFLUENCE_USER,
      password: process.env.WORDPRESS_PASSWORD,
      endpoint: process.env.WORDPRESS_ENDPOINT,
      spaceKey: process.env.CONFLUENCE_SPACE_KEY,
      rootPageId: process.env.CONFLUENCE_ROOT_PAGE_ID, // 可选
    },
    wordpress: {
      username: process.env.WORDPRESS_USERNAME,
      password: process.env.WORDPRESS_PASSWORD,
      endpoint: process.env.WORDPRESS_ENDPOINT,
    }
  },
  image: {
    enable: true,
    platform: 'local',
    local: {
      outputDir: './docs/images',
      prefixKey: '/images',
      pathFollowDoc: false,
    },
    oss: {
      secretId: process.env.OSS_SECRET_ID,
      secretKey: process.env.OSS_SECRET_KEY,
      bucket: process.env.OSS_BUCKET,
      region: process.env.OSS_REGION,
      host: process.env.OSS_HOST,
      prefixKey: '',
    },
    cos: {
      secretId: process.env.COS_SECRET_ID,
      secretKey: process.env.COS_SECRET_KEY,
      bucket: process.env.COS_BUCKET,
      region: process.env.COS_REGION,
      host: process.env.COS_HOST,
      prefixKey: '',
    },
    qiniu: {
      secretId: process.env.QINIU_SECRET_ID,
      secretKey: process.env.QINIU_SECRET_KEY,
      bucket: process.env.QINIU_BUCKET,
      region: process.env.QINIU_REGION,
      host: process.env.QINIU_HOST,
      prefixKey: '',
    },
    upyun: {
      user: process.env.UPYUN_USER,
      password: process.env.UPYUN_PASSWORD,
      bucket: process.env.UPYUN_BUCKET,
      host: process.env.UPYUN_HOST,
      prefixKey: '',
    },
    github: {
      token: process.env.GITHUB_TOKEN,
      user: process.env.ELOG_GITHUB_USER,
      repo: process.env.ELOG_GITHUB_REPO,
      prefixKey: '',
    }
  }
}

发生了什么?

飞书图片下载 api 请求过于频繁之后会返回一个报错的 json,elog 将这个 json 的内容视为了一张图片,并按照保存图片的逻辑保存了 json,导致出现了问题

image image

Elog错误日志

没有产生错误日志

附加信息

No response

LetTTGACO commented 6 days ago

好的,我会尝试复现一下。

LetTTGACO commented 2 days ago

查了一下飞书文档频控策略:飞书对于机器人的API 限制频率为:100 次/分钟,5 次/秒。

我在elog 中加入 limit 属性进行并发限制,可以尝试一下,如果还是经常出现并发问题。我会考虑按照飞书文档给出的解决办法进行处理

请升级到最新 beta 版本试一下:0.14.2-beta.1

默认限制 limit 为 3,即同时请求 3 次,如果依然遇到错误,可以尝试调低limit来控制并发请求。

limit 数值越低,意味着从飞书下载图片的时长会越长,会导致初次同步时,整个同步流程时长变长

image: {
enable: true,
platform: 'local',
limit: 3,
local: {
outputDir: './docs/images',
prefixKey: '/images',
pathFollowDoc: false,
}
}
xiehuacheng commented 1 day ago

刚刚升级并尝试了,之前不能正常下载的都恢复正常了,万分感谢,辛苦🙇‍♂️