Jeff-Tian / serverless-space

万能 BFF:这是我利用 AWS 免费资源构建一个 SFF 的实验场地,用于验证一些 Serverless 的想法。
https://graph.pa-ca.me/
15 stars 4 forks source link

微信公众号带有视频的情况会发布失败:501 #91

Open Jeff-Tian opened 8 months ago

Jeff-Tian commented 8 months ago

2023-12-27T10:42:58.811Z    9fbe67fc-b047-580a-9d32-39fab19a5e8b    ERROR   Invoke Error    {     "message": "Request failed with status code 501",     "name": "AxiosError",     "stack": "AxiosError: Request failed with status code 501\n    at settle (/opt/nodejs/node_modules/axios/dist/node/axios.cjs:1967:12)\n    at IncomingMessage.handleStreamEnd (/opt/nodejs/node_modules/axios/dist/node/axios.cjs:3062:11)\n    at IncomingMessage.emit (node:events:529:35)\n    at endReadableNT (node:internal/streams/readable:1368:12)\n    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)",     "config": {         "transitional": {             "silentJSONParsing": true,             "forcedJSONParsing": true,             "clarifyTimeoutError": false         },         "adapter": [             "xhr",             "http"         ],         "transformRequest": [             null         ],         "transformResponse": [             null         ],         "timeout": 0,         "xsrfCookieName": "XSRF-TOKEN",         "xsrfHeaderName": "X-XSRF-TOKEN",         "maxContentLength": -1,         "maxBodyLength": -1,         "env": {},         "headers": {             "Accept": "application/json, text/plain, */*",             "Content-Type": "application/json",             "User-Agent": "axios/1.6.3",             "Content-Length": "16917",             "Accept-Encoding": "gzip, compress, deflate, br"         },         "method": "post",         "url": "https://api.weixin.qq.com/cgi-bin/draft/add?access_token=75_r12-40QK0rTPb7eYIZ93jeq53qrk4QZ7L4LzV6NKQ0sTJGsfSUS-YNU70TxZHRXfexVy3YcHLJLUhbOQWk4-JDVEQEXGL5VsTpozZRWGWR-j7Ib-FggTfXdofkIAANgAFATTG",         "data": "{\"articles\":[{\"title\":\"借助 Bedrock SDK,全面释放内容生产力\",\"author\":\"哈德韦\",\"content\":\"<blockquote>\\n<p>未来是内容为王的时代,就连房地产行业,也要靠内容才更有机会了。<br>\\n—— Jeff Tian</p>\\n</blockquote>\\n<p>既然内容如此重要,那么作为技术从业者,可以为此做些什么呢?我想,通过大模型的力量,将内容生产效率提高到下一个层次,会是一个凸显技术价值的有趣尝试。<br />本篇文章将介绍一下在无头内容管理系统 Strapi 中集成宇宙最强富文编辑器 CKEditor 5,并添加 AI 小助手的过程,希望在内容生产上助你一臂之力!<br />另外,我在上一篇文章《<a href=\\\"%5Bhttps://zhuanlan.zhihu.com/p/674125115%5D(https://zhuanlan.zhihu.com/p/674125115)\\\">欢迎来调戏我:在公众号里对接 AWS Bedrock 服务 - Jeff Tian的文章 - 知乎</a> 》中介绍了如何将 AI 小助手接入微信公众号,但沮丧的是,听说这样做会极有可能被微信官方封号:<br /><img src=\\\"https://cdn.nlark.com/yuque/0/2023/png/221736/1703657510695-84ba9fcb-ac50-4564-9b1d-60fb16c20afd.png#averageHue=%23f3f2f2&amp;clientId=u8dba5f59-1967-4&amp;from=paste&amp;height=134&amp;id=u5afb3871&amp;originHeight=268&amp;originWidth=2056&amp;originalType=binary&amp;ratio=2&amp;rotation=0&amp;showTitle=false&amp;size=100833&amp;status=done&amp;style=none&amp;taskId=u4105f0e4-d7f9-484c-ae59-3ffca50a711&amp;title=&amp;width=1028\\\" alt=\\\"image.png\\\"><br />相比用在微信公众号的自动回复,本篇文章的应用场景,会更加具有可行性!<br>\\n<a name=\\\"GIL8C\\\"></a></p>\\n<h1>在线体验</h1>\\n<p>https://strapi.brickverse.dev/admin<br /><img src=\\\"https://cdn.nlark.com/yuque/0/2023/gif/221736/1703658038083-2c2d1bb9-8255-4af5-b70d-7641fbf61737.gif#averageHue=%23fbfbfb&amp;clientId=u8dba5f59-1967-4&amp;from=ui&amp;id=ue12ff58f&amp;originHeight=808&amp;originWidth=2680&amp;originalType=binary&amp;ratio=2&amp;rotation=0&amp;showTitle=false&amp;size=10481668&amp;status=done&amp;style=none&amp;taskId=ucae07f0a-8dc9-4e3f-a336-0255bca2e83&amp;title=\\\" alt=\\\"dx8xj_d1x5l_b2e387090c.gif\\\"><br /><img src=\\\"https://cdn.nlark.com/yuque/0/2023/png/221736/1703658978193-5f0e00c2-bed7-4f69-ba32-b117e07f6134.png#averageHue=%238cb7aa&amp;clientId=u8dba5f59-1967-4&amp;from=paste&amp;height=528&amp;id=uf22ec6a3&amp;originHeight=1055&amp;originWidth=1920&amp;originalType=binary&amp;ratio=2&amp;rotation=0&amp;showTitle=false&amp;size=216878&amp;status=done&amp;style=none&amp;taskId=u7bfa1e1c-da47-4e92-b3fc-2c59e109821&amp;title=&amp;width=960\\\" alt=\\\"WX20231219-173453.png\\\"><br /><img src=\\\"https://cdn.nlark.com/yuque/0/2023/png/221736/1703658961917-7d943722-bd87-4391-9602-75c41d3e68de.png#averageHue=%2388a399&amp;clientId=u8dba5f59-1967-4&amp;from=paste&amp;height=528&amp;id=u39b21533&amp;originHeight=1055&amp;originWidth=1920&amp;originalType=binary&amp;ratio=2&amp;rotation=0&amp;showTitle=false&amp;size=319655&amp;status=done&amp;style=none&amp;taskId=uca24c258-0ad9-48f3-87a6-0845efe42ee&amp;title=&amp;width=960\\\" alt=\\\"WX20231219-173608.png\\\"><br /><img src=\\\"https://cdn.nlark.com/yuque/0/2023/png/221736/1703658694578-b251f4bf-3ace-4319-9526-ab41df86d671.png#averageHue=%23e9ebe4&amp;clientId=u8dba5f59-1967-4&amp;from=paste&amp;height=521&amp;id=ude9c52e8&amp;originHeight=1041&amp;originWidth=1885&amp;originalType=binary&amp;ratio=2&amp;rotation=0&amp;showTitle=false&amp;size=871508&amp;status=done&amp;style=none&amp;taskId=u5dd16b7c-713a-4db1-8224-d9f0cd99d5c&amp;title=&amp;width=942.5\\\" alt=\\\"image.png\\\"><br />其实不仅可以提示它生成内容文案,而且还可以让它帮忙优化格式。当然,一些常用的使用场景,比如内容翻译,都有快捷方式:<br /><img src=\\\"https://cdn.nlark.com/yuque/0/2023/png/221736/1703659493573-57c2f243-fca6-4146-a5fc-21f40f4a2380.png#averageHue=%23b1b476&amp;clientId=u8dba5f59-1967-4&amp;from=paste&amp;height=492&amp;id=u2931e915&amp;originHeight=984&amp;originWidth=1128&amp;originalType=binary&amp;ratio=2&amp;rotation=0&amp;showTitle=false&amp;size=125110&amp;status=done&amp;style=none&amp;taskId=uc27d5b9d-44b4-4df4-ba1b-35ee0a29238&amp;title=&amp;width=564\\\" alt=\\\"image.png\\\"><br>\\n<a name=\\\"me050\\\"></a></p>\\n<h1>Strapi</h1>\\n<p>Strapi 是一个优秀的无头内容管理系统,更多介绍详见《<a href=\\\"%5Bhttps://zhuanlan.zhihu.com/p/669725918%5D(https://zhuanlan.zhihu.com/p/669725918)\\\">给 Strapi Admin Portal 添加单点登录方式 - Jeff Tian的文章 - 知乎</a> 》。<br>\\n<a name=\\\"B5LYR\\\"></a></p>\\n<h1>CKEditor 5</h1>\\n<p>我认为它是宇宙最强富文本编辑器,Strapi 默认的文本编辑器,功能非常简单。通过加入 CKEditor 5,可以说直接在 Strapi 里集成了一个 word,已经无敌了。<br />更多对 CKEditor 5 的讨论,可以参考:《<a href=\\\"%5Bhttps://zhuanlan.zhihu.com/p/663815578%5D(https://zhuanlan.zhihu.com/p/663815578)\\\">对 UmiJS 和 ckeditor 的折腾 - Jeff Tian的文章 - 知乎</a> 》以及这篇回答:《<a href=\\\"%5Bhttps://www.zhihu.com/question/38699645/answer/2692905552%5D(https://www.zhihu.com/question/38699645/answer/2692905552)\\\">为什么都说富文本编辑器是天坑? - Jeff Tian的回答 - 知乎</a> 》。<br>\\n<a name=\\\"KTuuu\\\"></a></p>\\n<h1>AI 助手</h1>\\n<p>这其实还是 CKEditor 5 团队开发的,有了 AI 助手,那就是在宇宙最强的基础上更加如虎添翼了!不过,虽然 CKEditor 5 本身是免费的,但 AI 助手却不是免费的(许可证一年需要 5000 美元!)。好在有 30 天的试用期,我目前就是拿到了试用序列号。<br /><img src=\\\"https://cdn.nlark.com/yuque/0/2023/png/221736/1703659586599-3f0d5d68-9b7a-48ac-a72d-a46527056443.png#averageHue=%23d9e3c2&amp;clientId=u8dba5f59-1967-4&amp;from=paste&amp;height=933&amp;id=u2953cd54&amp;originHeight=1866&amp;originWidth=1728&amp;originalType=binary&amp;ratio=2&amp;rotation=0&amp;showTitle=false&amp;size=371805&amp;status=done&amp;style=none&amp;taskId=u69a9109b-8cf3-4586-bb01-d2396d751aa&amp;title=&amp;width=864\\\" alt=\\\"image.png\\\"><br>\\n<a name=\\\"so0ZN\\\"></a></p>\\n<h1>通过插件的方式将 CKEditor 5 以及 AI 小助手集成到 Strapi</h1>\\n<p>CKEditor 团队开发了 Strapi 插件,可以在 Strapi 里添加自定义字段,并启用 CKEditor 5。但是官方的插件,并不包含 AI 小助手,于是我 fork 了官方的插件,做了一些修改,将 AI 小助手添加进来了,源代码见:<a href=\\\"https://github.com/Jeff-Tian/strapi-plugin-ckeditor\\\">https://github.com/Jeff-Tian/strapi-plugin-ckeditor</a> 。<br>\\n<a name=\\\"exA6V\\\"></a></p>\\n<h2>使用方式</h2>\\n<p>在 Strapi 项目里:</p>\\n<pre><code class=\\\"language-json\\\">yarn add @jeff-tian/strapi-plugin-ckeditor\\nyarn build\\nyarn develop\\n</code></pre>\\n<p>配置也非常简单,只需要配置一下 CKEditor 5 的许可证即可(其他选项可以留空不配置)。<br /><img src=\\\"https://cdn.nlark.com/yuque/0/2023/png/221736/1703659803737-f6c70cb5-73eb-45fa-afdb-24b004a70c31.png#averageHue=%232a2a3a&amp;clientId=u8dba5f59-1967-4&amp;from=paste&amp;height=925&amp;id=u2d5c7612&amp;originHeight=1850&amp;originWidth=3112&amp;originalType=binary&amp;ratio=2&amp;rotation=0&amp;showTitle=false&amp;size=406169&amp;status=done&amp;style=none&amp;taskId=u4b9a8b4a-7320-4111-8852-e710068bed8&amp;title=&amp;width=1556\\\" alt=\\\"image.png\\\"><br>\\n<a name=\\\"uiEz8\\\"></a></p>\\n<h2>对接 Bedrock</h2>\\n<p>以上截图还展示了 AWS 的配置,这就是为了对接 Bedrock 服务,详见《<a href=\\\"%5Bhttps://zhuanlan.zhihu.com/p/674125115%5D(https://zhuanlan.zhihu.com/p/674125115)\\\">欢迎来调戏我:在公众号里对接 AWS Bedrock 服务 - Jeff Tian的文章 - 知乎</a> 》中对 Bedrock 的相关解释。<br />这样配置后,就可以使用 AI 小助手了!</p>\\n<p><a name=\\\"CPyHg\\\"></a></p>\\n<h1>通过后端方式为 CKEditor 5 的 AI 助手提供服务</h1>\\n<p>以上通过配置的方式来使用 Bedrock 服务,虽然简单,但是不推荐!如果通过配置 AWS 访问密钥,来从客户端调用 Bedrock 服务,就会将 AWS 的密钥暴露在前端,所以非常建议通过后端接口来为 CKEditor 5 的 AI 助手提供服务。尽管在前一篇文章《<a href=\\\"%5Bhttps://zhuanlan.zhihu.com/p/674125115%5D(https://zhuanlan.zhihu.com/p/674125115)\\\">欢迎来调戏我:在公众号里对接 AWS Bedrock 服务 - Jeff Tian的文章 - 知乎</a> 》中写的接口,完全可以用在这里,但是对于 AI 写作,使用流的方式,用户体验更佳,这样就可以看见它的“实时”写作过程。<br>\\n<a href=\\\"\\\"><img src=\\\"https://gw.alipayobjects.com/mdn/prod_resou/afts/img/A*NNs6TKOR3isAAAAAAAAAAABkARQnAQ\\\" alt=\\\"Screen Recording 2023-12-19 at 11.18.16.mov (32MB)\\\"></a><br>\\n后端接口可以写在任何地方,但既然我们的 Strapi 本身就是一个后端,不妨直接在 Strapi 里添加一个接口,用来为 AI 小助手服务。<br>\\n<a name=\\\"DGZFV\\\"></a></p>\\n<h2>添加一个方法调用 bedrock 服务</h2>\\n<p>这个文件的<strong>主要内容</strong>和上一篇《<a href=\\\"%5Bhttps://zhuanlan.zhihu.com/p/674125115%5D(https://zhuanlan.zhihu.com/p/674125115)\\\">欢迎来调戏我:在公众号里对接 AWS Bedrock 服务 - Jeff Tian的文章 - 知乎</a> 》中的 bedrock.js 文件内容几乎一模一样,唯一的区别是这里支持流的响应方式。</p>\\n<pre><code class=\\\"language-json\\\">const {\\n  BedrockRuntimeClient,\\n  InvokeModelWithResponseStreamCommand, InvokeModelCommand,\\n} = require(&quot;@aws-sdk/client-bedrock-runtime&quot;);\\n\\n// 心跳间隔设置为 15 秒,小于 Heroku 的 30 秒超时\\nconst heartbeatInterval = 15000;\\n\\nconst sleep = async (ms) =&gt; new Promise(resolve =&gt; setTimeout(resolve, ms))\\n\\nconst setupHeartBeat = (ctx, heartbeatInterval) =&gt; {\\n  const heartBeatFunc = () =&gt; {\\n    // 发送一个空白的数据块作为心跳\\n    if (!ctx.res.writableEnded) {\\n      ctx.res.write(JSON.stringify({&quot;completion&quot;: &quot;&quot;}));\\n    }\\n  }\\n\\n  return setInterval(heartBeatFunc, heartbeatInterval);\\n}\\n\\nmodule.exports = {\\n  async index(ctx, next) { // called by GET /hello\\n    ctx.body = 'Hello World!'; // we could also send a JSON\\n  },\\n\\n  async post(ctx, next) { // called by POST /hello\\n    console.log('ctx.request.body = ', ctx.request.body);\\n\\n    const input = {\\n      modelId: ctx.request.body.model ?? 'anthropic.claude-v2',\\n      contentType: 'application/json',\\n      accept: 'application/json',\\n      body: JSON.stringify({\\n        prompt: ctx.request.body.prompt,\\n        max_tokens_to_sample: ctx.request.body.max_tokens_to_sample ?? 2000,\\n        temperature: ctx.request.body.temperature ?? 1,\\n        top_p: ctx.request.body.top_p ?? 1,\\n        top_k: ctx.request.body.top_k ?? 250\\n      })\\n    }\\n\\n    const client = new BedrockRuntimeClient({\\n      region: &quot;us-east-1&quot;,\\n      &quot;credentials&quot;: {\\n        &quot;accessKeyId&quot;: &quot;from env&quot;,\\n        &quot;secretAccessKey&quot;: &quot;from env&quot;\\n      }\\n    });\\n\\n    async function stream() {\\n      // 不要让 Koa 自动处理响应\\n      ctx.respond = false;\\n      let heartbeat = null;\\n\\n      try {\\n        // 设置 HTTP 响应头\\n        ctx.res.writeHead(200, {\\n          'Content-Type': 'application/json',\\n          'Transfer-Encoding': 'chunked'\\n        });\\n\\n        heartbeat = setupHeartBeat(ctx, heartbeatInterval);\\n\\n        const command = new InvokeModelWithResponseStreamCommand(input);\\n        const res = await client.send(command);\\n\\n        for await (const event of res.body) {\\n          // 如果有心跳,清除它,因为我们即将发送实际数据\\n          if (heartbeat) {\\n            clearInterval(heartbeat);\\n            heartbeat = null;\\n          }\\n\\n          if (event.chunk &amp;&amp; event.chunk.bytes) {\\n            // 将 JSON 对象转换为字符串,并发送一个 JSON 块\\n            if (!ctx.res.writableEnded) {\\n              const response = Buffer.from(event.chunk.bytes).toString(&quot;utf-8&quot;) + &quot;\\\\n&quot;;\\n              console.log('response to client: ', response);\\n              ctx.res.write(response);\\n\\n              await sleep(600);\\n            }\\n          } else if (\\n            event.internalServerException \|\|\\n            event.modelStreamErrorException \|\|\\n            event.throttlingException \|\|\\n            event.validationException\\n          ) {\\n            console.error(event);\\n            if (!ctx.res.writableEnded) {\\n              ctx.res.write(JSON.stringify({\\n                &quot;completion&quot;: `Error: ${event.internalServerException?.message ?? event.modelStreamErrorException?.message ?? event.throttlingException?.message ?? event.validationException?.message}`\\n              }));\\n            }\\n\\n            break;\\n          }\\n\\n          if (!heartbeat) {\\n            // 数据发送后重新启动心跳\\n            heartbeat = setupHeartBeat(ctx, heartbeatInterval);\\n          }\\n        }\\n\\n        // 结束 HTTP 响应前清除心跳\\n        if (heartbeat) {\\n          clearInterval(heartbeat);\\n          heartbeat = null;\\n        }\\n\\n        ctx.res.end();\\n      } catch (ex) {\\n        console.error('bedrock error = ', ex);\\n        if (heartbeat) {\\n          clearInterval(heartbeat);\\n          heartbeat = null;\\n        }\\n        ctx.res.end(JSON.stringify({\\n          &quot;completion&quot;: `&lt;p&gt;抱歉,连接 Bedrock 出错了,原因是: ${ex.message}&lt;/p&gt;`,\\n          &quot;stop_reason&quot;: &quot;stop_sequence&quot;,\\n          &quot;stop&quot;: &quot;\\\\n\\\\nHuman:&quot;\\n        }));\\n      }\\n    }\\n\\n    async function nonStream() {\\n      ctx.set('Content-Type', 'application/json');\\n\\n      try {\\n        const command = new InvokeModelCommand(input);\\n        const response = await client.send(command);\\n\\n        console.log(&quot;-------------------&quot;);\\n        console.log(&quot;---Full Response---&quot;);\\n        console.log(&quot;-------------------&quot;);\\n        console.log(response);\\n\\n        const rawRes = response.body;\\n        const jsonString = new TextDecoder().decode(rawRes);\\n\\n        console.log(&quot;-------------------------&quot;);\\n        // Answers are in parsedResponse.completion\\n        console.log(jsonString);\\n        console.log(&quot;-------------------------&quot;);\\n\\n        ctx.body = jsonString;\\n      } catch (ex) {\\n        console.error('bedrock error = ', ex);\\n\\n        ctx.body = {\\n          &quot;completion&quot;: `&lt;p&gt;抱歉,连接 Bedrock 出错了,原因是: ${ex.message}&lt;/p&gt;`\\n        }\\n      }\\n    }\\n\\n    if (ctx.request.body.stream) {\\n      await stream();\\n    } else {\\n      await nonStream();\\n    }\\n  }\\n};\\n\\n</code></pre>\\n<p><a name=\\\"z7QTF\\\"></a></p>\\n<h2>克服 Heroku 的响应超时限制</h2>\\n<p>在《<a href=\\\"%5Bhttps://zhuanlan.zhihu.com/p/669725918%5D(https://zhuanlan.zhihu.com/p/669725918)\\\">给 Strapi Admin Portal 添加单点登录方式 - Jeff Tian的文章 - 知乎</a> 》中我说过,部署 Strapi 的最简单的方式就是将它部署到 Heroku 这个 PaaS 平台,但是 Heroku 对一个请求响应,却有 30 秒的限制。一旦和 Bedrock 服务的网络连接稍慢,就会造成一个 AI 帮写的停滞,在这时,尽管可以通过重试,有极大的概率成功写完,但是如果能够减少这种情况,还是要尽量减少。<br />因此,你会在上面的代码中,看到一些所谓的心跳响应。是的,本来是不必要的,但是在 Heroku 的基础设施上,不得已而为之,写了一个定时返回空响应的代码。代码虽丑,但效果拔群!<br>\\n<a name=\\\"Ec1dy\\\"></a></p>\\n<h1>最终效果视频演示</h1>\\n<p><a href=\\\"\\\"><img src=\\\"https://gw.alipayobjects.com/mdn/prod_resou/afts/img/A*NNs6TKOR3isAAAAAAAAAAABkARQnAQ\\\" alt=\\\"Screen Recording 2023-12-19 at 10.53.39.mov (151.64MB)\\\"></a><a name=\\\"ARfwE\\\"></a></p>\\n<h1>彩蛋</h1>\\n<p>最后附上 AWS Bedrock 的更多介绍资料:<br /><a href=\\\"https://www.yuque.com/attachments/yuque/0/2023/pdf/221736/1703663683690-dd996105-325c-4556-a6bc-06659d917e9f.pdf\\\">GCR Amazon Bedrock First Call Deck CN.pdf</a></p>\\n<mp-miniprogram data-miniprogram-appid=\\\"wx8c777d630f2b78e3\\\" data-miniprogram-path=\\\"pages/yuque/article?slug=ad2ha2vntysmw8f6\\\" data-miniprogram-title=\\\"在小程序中查看本文\\\" data-miniprogram-imageurl=\\\"http://mmbiz.qpic.cn/mmbiz_jpg/AhHNmKXia2glU45JGpDl7LC5ovAfU9GoNYQ9tWZiak2pdQEgNmNxuib1xdL41Cyn0yBdH5Q1ZpyWGapvcKehY2XCA/0\\\"></mp-miniprogram>\",\"content_source_url\":\"https://jeff-tian.jiwai.win/posts/ad2ha2vntysmw8f6/\",\"need_open_comment\":1,\"thumb_media_id\":\"PiIEZ-GBv-fmg8_8wVQ_0b4Nmo-rmMhKaQACc5veI-iJgu1vbQqbCTGuKpLbndsD\"}]}"     },     "code": "ERR_BAD_RESPONSE",     "status": 501 } | 2023-12-27T18:43:00.860+08:00
-- | --
Jeff-Tian commented 1 week ago

另外,在 content 中带有 CREATE DATABASE 或者 DROP DATABASE 的字眼,也会触发 501 。

curl --location 'https://api.weixin.qq.com/cgi-bin/draft/add?access_token=Rf3ffELHjup3k9gzj1HHzMUutBtmAgubJtyXQ0HxATbf-T9SqrWyXICBNeAHATSE' \
--header 'Content-Type: application/json' \
--data '{
            "articles": [
                {
                    "title": "test",
                    "author": "哈德韦",
                    "content": "test<p>docker 真是好,docker compose 更是不错。然而,在频繁的启动关闭中,有时候也很容易让人烦,比如很多测试需要启动一些像数据库之类的依赖,就不得不在 README 里提一句,跑测试前需要先 <code>docker compose up -d</code> 一下。虽然似乎很简单,但仍然有点麻烦,不免会让人在此栽跟着,连大佬也不能幸免,比如:</p>\n<p><img src=\"http://mmbiz.qpic.cn/mmbiz_png/AhHNmKXia2gnATu1yIWGFvInk2er3ooribW7jhk0ymllic39IzhfXKiaXT8tJwcIiaVyA8wmQI9RYy3I2j4orl9EHMw/0?wx_fmt=png\" alt=\"\"></p><p>我发现通过 test containers 可以不用再手动去执行这些命令。</p><p>感觉上,test containers 就是一个全托管的 docker compose,即只需要写测试代码就好了,至于启动容器和销毁容器,不用再手动做了。当然,底层还是容器,可以在发起执行测试的命令之后,通过 docker desktop 看到测试容器正在启动中:</p><p><img src=\"http://mmbiz.qpic.cn/mmbiz_png/AhHNmKXia2gnATu1yIWGFvInk2er3ooribxp7ic9XB8OBTGl8jUfJ04cOauaCcn8EukSic7RJPicNB1ZqHlxuZbewqQ/0?wx_fmt=png\" alt=\"\"></p><p>在测试运行结束后,可以看到容器被自动销毁:</p><p><img src=\"http://mmbiz.qpic.cn/mmbiz_png/AhHNmKXia2gnATu1yIWGFvInk2er3ooribeheElM4xplsDQ9wsUfTH0ATh8IZsalMWKkeyfrLgg4n4YIeJOUIRIg/0?wx_fmt=png\" alt=\"\"></p><p>这里分享一个完整的测试案例:应用程序启动了一个 pulsar 消费者,用来消费 pulsar 上某一主题上的事件消息,从而用来更新用户最近和系统互动的时间。要测试这种场景,就可以让测试跑起来之前,通过 test containers 运行一个 pulsar 服务,然后就可以在测试代码中使用真正的 pulsar 消费者,避免写一堆模拟客户端和消费者等等。</p><p>以 dotnet core 应用举例,首先需要写一个 TestingBase 类,做一些测试生命周期管理,并将使用 test containers 启动 pulsar 服务的代码放在其中:</p><pre><code class=\"language-java\">public abstract class TestingFixtureBase(string databaseName){    private readonly PulsarContainer _pulsar = new PulsarBuilder().WithImage(&quot;apachepulsar/pulsar:2.10.0&quot;).Build();    protected abstract IServiceProvider Services { get; }    protected IHostBuilder HostBuilder { get; private set; }    protected async Task Initialize()    {        await _pulsar.StartAsync();        HostBuilder = Program.CreateHostBuilder([]);        HostBuilder.UseEnvironment(&quot;local&quot;);        HostBuilder.UseSerilog();        ConfigureConfiguration(HostBuilder);        HostBuilder.ConfigureServices((hostContext, services) =&gt;        {            var dbConfig = hostContext.Configuration.Get&lt;DatabaseConfiguration&gt;()!;            RecreateTestDatabase(dbConfig.Host, dbConfig.Username, dbConfig.Password, dbConfig.DatabaseName);        });    }    private void RecreateTestDatabase(string host, string username, string password, string databaseName)    {        // 创建连接字符串时不要指定数据库名称,否则如果数据库不存在将抛出异常。        var connectionString = new MySqlConnectionStringBuilder        {            Server = host,            UserID = username,            Password = password,            Port = 3306,            // 在 GitHub Actions 中运行时需要            SslMode = MySqlSslMode.None,        };        using var connection = new MySqlConnection(connectionString.ConnectionString);        connection.Open();        using var dropCommand = new MySqlCommand($&quot;DROP DATABASE IF EXISTS `{databaseName}`&quot;, connection);        dropCommand.ExecuteNonQuery();        using var createCommand = new MySqlCommand($&quot;CREATE DATABASE `{databaseName}`&quot;, connection);        createCommand.ExecuteNonQuery();    }    private void ConfigureConfiguration(IHostBuilder hostBuilder)    {        hostBuilder.ConfigureAppConfiguration((_, config) =&gt;        {            var databaseHost = Environment.GetEnvironmentVariable(&quot;DATABASE_HOST&quot;) ?? &quot;localhost&quot;;            config.AddInMemoryCollection(new Dictionary&lt;string, string?&gt;            {                [&quot;DATABASE_HOST&quot;] = databaseHost,                [&quot;DATABASE_PORT&quot;] = &quot;3306&quot;,                [&quot;DATABASE_USERNAME&quot;] = &quot;root&quot;,                [&quot;DATABASE_PASSWORD&quot;] = &quot;localdb123&quot;,                [&quot;DATABASE_NAME&quot;] = databaseName,                [&quot;DATABASE_SSL_MODE&quot;] = nameof(MySqlSslMode.None),                [&quot;DATABASE_IAM_AUTHENTICATION_ENABLED&quot;] = &quot;false&quot;,                [&quot;DATABASE_CONNECTION_IDLE_TIMEOUT&quot;] = &quot;3600&quot;,                [&quot;DATABASE_CONNECTION_LIFETIME&quot;] = &quot;3600&quot;,                [&quot;PULSAR_CLIENT_ID&quot;] = &quot;fake-client-id&quot;,                [&quot;PULSAR_CLIENT_SECRET&quot;] = &quot;fake-client-secret&quot;,                [&quot;PULSAR_SERVICE_URL&quot;] = _pulsar.GetBrokerAddress(),                [&quot;TOPIC&quot;] = &quot;persistent://public/default/your-topic-name&quot;,                [&quot;SUBSCRIPTION_NAME&quot;] = &quot;your-sub-name&quot;,            });        });    }    protected async Task Dispose()    {        var dbContext = Services.CreateScope().ServiceProvider.GetRequiredService&lt;ApplicationDbContext&gt;();        await dbContext.Database.EnsureDeletedAsync();        await _pulsar.DisposeAsync().AsTask();    }}</code></pre><p>然后需要创建一个类继承自上面的 TestingBase,并实现 IAsyncLifetime 类,写一些测试通用的代码,用来做测试夹具:</p><pre><code class=\"language-csharp\">public class BackgroundJobFixture() : TestingFixtureBase(&quot;background_jobs_tests&quot;), IAsyncLifetime{    private IServiceProvider _serviceProvider;    protected override IServiceProvider Services =&gt; _serviceProvider;        public IHost Host;    public async Task InitializeAsync()    {        await Initialize();        Host = HostBuilder.Build();        _serviceProvider = Host.Services;        var migrator = _serviceProvider.GetRequiredService&lt;DatabaseMigrator&gt;();        migrator.ExecuteMigrations(true);    }    public Task DisposeAsync() =&gt; Dispose();    [CollectionDefinition(&quot;background_job_scenarios&quot;, DisableParallelization = false)]    public class BackgroundJobsScenarioCollection : ICollectionFixture&lt;BackgroundJobFixture&gt;;    [Collection(&quot;background_job_scenarios&quot;)]    public abstract class BackgroundJobsScenarioContext : IAsyncLifetime    {        private readonly IServiceScope _serviceScope;        private readonly IServiceProvider _serviceProvider;        protected BackgroundJobsScenarioContext(BackgroundJobFixture fixture)        {            _serviceScope = fixture.Services.CreateScope();            _serviceProvider = _serviceScope.ServiceProvider;        }        protected T GetService&lt;T&gt;() where T : class =&gt; _serviceProvider.GetRequiredService&lt;T&gt;();        public Task InitializeAsync()        {            return Task.CompletedTask;        }        public Task DisposeAsync()        {            _serviceScope.Dispose();            return Task.CompletedTask;        }    }}</code></pre><p>最后,测试可能像是长这样:</p><pre><code class=\"language-csharp\">public class EventListenerJobEnd2EndTests : BackgroundJobFixture.BackgroundJobsScenarioContext, IAsyncLifetime{    private readonly ApplicationDbContext _applicationDbContext;    private readonly IPulsarClient _client;    private readonly IProducer&lt;string&gt; _producer;    private readonly IHost _host;    public EventListenerJobEnd2EndTests(BackgroundJobFixture fixture) : base(fixture)    {        _host = fixture.Host;        _applicationDbContext = GetService&lt;ApplicationDbContext&gt;();        var configuration = GetService&lt;IConfiguration&gt;();        _client = PulsarClient.Builder()            .ServiceUrl(new Uri(configuration[&quot;PULSAR_SERVICE_URL&quot;]!))            .Build();        _producer = _client.NewProducer(Schema.String)            .Topic(configuration[&quot;TOPIC&quot;]!)            .Create();    }    public new async Task DisposeAsync()    {        await _producer.DisposeAsync();        await _client.DisposeAsync();        await base.DisposeAsync();    }    [Fact]    public async Task Should_ProcessEvent_WhenEventComes()    {        // arrange        var user = new UserBuilder().CreateValid().User;        _applicationDbContext.Users.Add(user);        await _applicationDbContext.SaveChangesAsync();        await _host.StartAsync();        var event            = (V1EnvelopeCloudEventBuilder.Create().WithPublicUserId(user.PublicId).Generate());        // act        _ = await _producer.Send(JsonSerializer.Serialize(event)).ConfigureAwait(true);        // 等待事件被处理。        await Task.Delay(2000);        // assert        var actual = _applicationDbContext.ApplicationUserActivities.FirstOrDefault(x =&gt; x.UserId == user.Id);        actual.Should().NotBeNull();        actual!.LastActivity.Should().BeCloseTo(loyaltyEvent.Data.Timestamp, TimeSpan.FromSeconds(1));    }",
                    "content_source_url": "https://jefftian.dev",
                    "need_open_comment": 1,
                    "thumb_media_id": "PiIEZ-GBv-fmg8_8wVQ_0b8gPhjXvzDXGRdvwIHm65K4zEVfmS-Gf2FS3Tj9_KfW"
                }
            ]
        }'

以上请求中,将 content 的 CREATE 和 DROP DATABASE 改掉即可成功。