上传文件慢

[longjuan]

用s3插件上传文件明显比alioss插件慢，日志显示从插件返回到插入数据库有两秒多的间隔。

2023-02-21T13:24:21.151+08:00 DEBUG 7156 --- [ctor-http-nio-3] r.h.app.plugin.SpringExtensionFactory    :   Extension class ' run.halo.s3os.S3OsAttachmentHandler' belongs to halo-plugin 'PluginS3ObjectStorage' and will be autowired by using its application context.
2023-02-21T13:24:21.151+08:00 DEBUG 7156 --- [ctor-http-nio-3] r.h.app.plugin.SpringExtensionFactory    :   Extension class ' run.halo.alioss.AliOssAttachmentHandler' belongs to halo-plugin 'PluginAliOSS' and will be autowired by using its application context.
2023-02-21T13:24:22.874+08:00  INFO 7156 --- [nc-response-0-2] run.halo.s3os.S3OsAttachmentHandler      : operation: createMultipartUpload, result: CreateMultipartUploadResponse(Bucket=test-1305034426, Key=image-afec.png, UploadId=16769570627a2683ca4900aef40325acec3637e9396e68766441093efa800a8442746ef9ac)
2023-02-21T13:24:22.990+08:00  INFO 7156 --- [nc-response-0-3] run.halo.s3os.S3OsAttachmentHandler      : operation: uploadPart, result: UploadPartResponse(ETag="7126c384effbb2fe8f709c9850c29a66")
2023-02-21T13:24:23.094+08:00  INFO 7156 --- [nc-response-0-4] run.halo.s3os.S3OsAttachmentHandler      : operation: completeUpload, result: CompleteMultipartUploadResponse(Location=http://test-1305034426.cos.ap-guangzhou.myqcloud.com/image-afec.png, Bucket=test-1305034426, Key=image-afec.png, ETag="2431c6ff6140ce32c954bf0a7341e1e6-1")
2023-02-21T13:24:23.164+08:00  INFO 7156 --- [nc-response-0-5] run.halo.s3os.S3OsAttachmentHandler      : operation: getMetadata, result: HeadObjectResponse(AcceptRanges=bytes, LastModified=2023-02-21T05:24:22Z, ContentLength=3367, ETag="2431c6ff6140ce32c954bf0a7341e1e6-1", ContentType=image/png, Metadata={})
2023-02-21T13:24:23.165+08:00  INFO 7156 --- [nc-response-0-5] run.halo.s3os.S3OsAttachmentHandler      : Upload object image-afec.png to bucket test-1305034426 successfully
2023-02-21T13:24:25.267+08:00 DEBUG 7156 --- [nc-response-0-5] r.h.a.extension.controller.DefaultQueue  : Adding request Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] immediately
2023-02-21T13:24:25.267+08:00 DEBUG 7156 --- [nc-response-0-5] r.h.a.extension.controller.DefaultQueue  : Adding request Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] after PT0.005S
2023-02-21T13:24:25.267+08:00 DEBUG 7156 --- [nc-response-0-5] r.h.a.extension.controller.DefaultQueue  : Added request Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] after PT0.005S
2023-02-21T13:24:25.282+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.extension.controller.DefaultQueue  : Take request Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] at 2023-02-21T05:24:25.282637900Z
2023-02-21T13:24:25.282+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.e.controller.DefaultController     : run.halo.app.core.extension.reconciler.attachment.AttachmentReconciler-worker-1 >>> Reconciling request Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] at 2023-02-21T05:24:25.282637900Z
2023-02-21T13:24:25.285+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.extension.controller.DefaultQueue  : Adding request Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] immediately
2023-02-21T13:24:25.285+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.extension.controller.DefaultQueue  : Adding request Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] after PT0.005S
2023-02-21T13:24:25.285+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.c.e.r.a.AttachmentReconciler       : Set permalink https://test.cos.zway.top/image-afec.png for attachment 9dd2a0e4-05d4-41a4-86ef-38472f2d5800
2023-02-21T13:24:25.288+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.extension.controller.DefaultQueue  : Adding request Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] immediately
2023-02-21T13:24:25.288+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.extension.controller.DefaultQueue  : Adding request Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] after PT0.005S
2023-02-21T13:24:25.288+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.e.controller.DefaultController     : run.halo.app.core.extension.reconciler.attachment.AttachmentReconciler-worker-1 >>> Reconciled request: Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] with result: null
2023-02-21T13:24:25.298+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.extension.controller.DefaultQueue  : Take request Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] at 2023-02-21T05:24:25.298645700Z
2023-02-21T13:24:25.298+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.e.controller.DefaultController     : run.halo.app.core.extension.reconciler.attachment.AttachmentReconciler-worker-1 >>> Reconciling request Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] at 2023-02-21T05:24:25.298645700Z
2023-02-21T13:24:25.299+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.c.e.r.a.AttachmentReconciler       : Set permalink https://test.cos.zway.top/image-afec.png for attachment 9dd2a0e4-05d4-41a4-86ef-38472f2d5800
2023-02-21T13:24:25.300+08:00 DEBUG 7156 --- [tReconciler-t-1] r.h.a.e.controller.DefaultController     : run.halo.app.core.extension.reconciler.attachment.AttachmentReconciler-worker-1 >>> Reconciled request: Request[name=9dd2a0e4-05d4-41a4-86ef-38472f2d5800] with result: null

[longjuan]

是关闭client时，需要阻塞两秒才能关闭，详见： https://github.com/aws/aws-sdk-java-v2/blob/71cfc0fee697b7ac29a30135cff461cf65948781/http-clients/netty-nio-client/src/main/java/software/amazon/awssdk/http/nio/netty/NettyNioAsyncHttpClient.java#L195-L215

后续通过异步方式关闭client

[JohnNiang]

从源码和 https://github.com/aws/aws-sdk-java-v2/pull/1069 来看，2s 似乎只是超时时间，而不是 sleep 2s 再关闭吧。如果我理解错误，请纠正我。

这里如果使用 S3Client 会不会比 S3AsyncClient 更好呢？

[JohnNiang]

从源码和 aws/aws-sdk-java-v2#1069 来看，2s 似乎只是超时时间，而不是 sleep 2s 再关闭吧。如果我理解错误，请纠正我。

这里如果使用 S3Client 会不会比 S3AsyncClient 更好呢？

这里我可能理解错误了，确实是会等待 2s 静默期才去关闭。

请问当初为什么选择 S3AsyncClient 呢？从PR https://github.com/halo-sigs/plugin-s3/pull/7 中我没有找到对应的原因。

[longjuan]

请问当初为什么选择 S3AsyncClient 呢？从PR #7 中我没有找到对应的原因。

当时看到s3在webflux中使用的示例就是使用的异步客户端，就用了。

[longjuan]

异步客户端的关闭确实是个问题，需要换成同步客户端吗，还是这样子异步关闭就行。

[JohnNiang]

异步客户端的关闭确实是个问题，需要换成同步客户端吗，还是这样子异步关闭就行。

有兴趣的话，可以看看 https://github.com/halo-sigs/plugin-s3/compare/main...JohnNiang:plugin-s3:refactor/use-s3client?expand=1，也非常欢迎手动测试测试。我尝试修改为了同步 Client，这也有可能解决了 https://github.com/halo-sigs/plugin-s3/issues/21。

[longjuan]

https://github.com/halo-sigs/plugin-s3/compare/main...JohnNiang:plugin-s3:refactor/use-s3client?expand=1

这个代码我处理了checkFileExistsAndRename的NoSuchKeyException后仍然跑不通，出现java.lang.IllegalStateException: Connection pool shut down，明天我再看看

[longjuan]

@JohnNiang 改了几行调通了，https://github.com/JohnNiang/plugin-s3/pull/1

[longjuan]

提PR吗

[JohnNiang]

Hi @longjuan ，近期我将重构一下代码后提交 PR。

/assign