search

apache / apisix-java-plugin-runner

APISIX Plugin Runner in Java
https://apisix.apache.org/
Apache License 2.0
131 stars 98 forks source link

bug: 通过网关上传附件，16m左右会报错 #198

Open dargoner opened 2 years ago

dargoner commented 2 years ago

Issue description

Environment

Minimal test code / Steps to reproduce the issue

  1. 使用POSTMAN模拟上传附件
  2. 附件大小大于16M报 503错误

What's the actual result? (including assertion message & call stack if applicable)

What's the expected result?

看到 plugins/ext-plugin/init.lua 有个限制

image

调大这个MAX_DATA_SIZE限制，好像无效，继续报错，暂时无解

tzssangglass commented 2 years ago

调大这个MAX_DATA_SIZE限制，好像无效，继续报错，暂时无解

What are the error logs about this? Are you observing errors generated by this limitation?

dargoner commented 2 years ago

image

调大这个MAX_DATA_SIZE限制，好像无效，继续报错，暂时无解

What are the error logs about this? Are you observing errors generated by this limitation?

dargoner commented 2 years ago

image

dargoner commented 2 years ago

ext-plugin/init.lua image postman test image apisix error image

tzssangglass commented 2 years ago

So that's the limit. After you adjust MAX_DATA_SIZE, you say it doesn't work, are there some error logs about it?

dargoner commented 2 years ago

So that's the limit. After you adjust MAX_DATA_SIZE, you say it doesn't work, are there some error logs about it?

image plugin error

dargoner commented 2 years ago

调整为:MAX_DATA_SIZE=1024x1024x110

image

tzssangglass commented 2 years ago

调整为:MAX_DATA_SIZE=1024x1024x110

OK it looks like the plugin runner doesn't handle exceeding MAX_DATA_SIZE very well at the moment. plugin runner is not designed for such a large body scenario. But I don't think it matters now. Can I know your scenario? When would you need to pass a body larger than 16 M to the plugin runner?

dargoner commented 2 years ago

主要业务场景需要的，上传附件比较大，如扫描的影像件（一起压缩后也很大）等，也不是要能无限大，能调整到100-200m左右应该也够了。

dargoner commented 2 years ago

是不是netty什么限制

tzssangglass commented 2 years ago

I will track this issue

tzssangglass commented 2 years ago

After verification, I think the problem is related to the maximum size of the StringBuilder. ref: https://stackoverflow.com/questions/38067717/how-many-characters-can-a-java-stringbuilder-hold

https://github.com/apache/apisix-java-plugin-runner/blob/d2fe045624fad0b7162b11b8acf836d724f870ea/runner-plugin-sdk/src/main/java/org/apache/apisix/plugin/runner/ExtraInfoResponse.java#L38-L43

100M file over the max size of StringBuilder

dargoner commented 2 years ago

MAX_DATA_SIZE 调大到100，但是上传30m附件也会报错的

tzssangglass commented 2 years ago

MAX_DATA_SIZE 调大到100，但是上传30m附件也会报错的

As mentioned above, this is a limitation of StringBuilder. It has nothing to do with MAX_DATA_SIZE. Maybe you need to change the source code here and instead of using StringBuilder, read this.resp.result(i) as a file stream