bug: failed to get large body

zdzh commented 1 year ago

Issue description

我用request.Body来获取请求的body，但是当请求请求体过大时，该函数会提示获取失败，数据被截断。

Environment

APISIX Go Plugin Runner's version: v0.3.0
APISIX version: 2.15.0
Go version: go version go1.15.1 linux/amd64
OS (cmd: uname -a): Linux fefcfe0549f3 4.18.0 #1 SMP Tue Sep 27 10:33:52 CST 2022 x86_64 x86_64 x86_64 GNU/Linux

Minimal test code / Steps to reproduce the issue

构造一个足够大的body，然后在go-runner中通过req.Body获取body

What's the actual result? (including assertion message & call stack if applicable)

传入的body首先提示存入一个临时文件，然后在调用req.Body时获取body失败，提示被截断。觉得是存入临时文件的数据没有正确传递给go-runner，但是在ext-plugin/init.lua查看body的长度好像没问题

2022/11/14 17:15:43 [warn] 85510#85510: *444 a client request body is buffered to a temporary file /usr/local/apisix/client_body_temp/0000000001, client: 10.131.115.28, server: _, request: "POST ******** HTTP/1.1", host: "127.0.0.1:9080"
2022/11/14 17:15:43 [warn] 85510#85510: *444 [lua] init.lua:319: handle_extra_info(): request body len 900938, client: 10.131.115.28, server: _, request: "POST ****** HTTP/1.1", host: "127.0.0.1:9080"
2022/11/14 17:15:43 [warn] 85514#85514: *49 [lua] init.lua:915: 2022-11-14T17:15:43.465+0800    INFO    http/request.go:390     receive rpc type: 3 data length: 900964
, context: ngx.timer
2022/11/14 17:15:43 [warn] 85514#85514: *49 [lua] init.lua:915: 2022-11-14T17:15:43.466+0800    ERROR   util/msg.go:55  read: truncated, only get the first 219260 bytes
github.com/apache/apisix-go-plugin-runner/internal/util.ReadErr
        /home/go/pkg/mod/github.com/apache/apisix-go-plugin-runner@v0.3.0/internal/util/msg.go:55
github.com/apache/apisix-go-plugin-runner/internal/http.(*Request).askExtraInfo
        /home/go/pkg/mod/github.com/apache/apisix-go-plugin-runner@v0.3.0/internal/http/request.go:394
github.com/apache/apisix-go-plugin-runner/internal/http.(*Request).Body
        /home/go/pkg/mod/github.com/apache/apisix-go-plugin-runner@v0.3.0/internal/http/request.go:183

What's the expected result?

获取body成功

soulbird commented 1 year ago

I'll check it out later

zdzh commented 1 year ago

func (r *Request) askExtraInfo(builder *flatbuffers.Builder,
    infoType ei.Info, info flatbuffers.UOffsetT) ([]byte, error) { 
       ......
    ty := header[0]
    header[0] = 0
    length := binary.BigEndian.Uint32(header)

    log.Infof("receive rpc type: %d data length: %d", ty, length)

    buf := make([]byte, length)
    n, err = c.Read(buf)
    if util.ReadErr(n, err, int(length)) {
        return nil, common.ErrConnClosed
    }
        ....
}

这里从conn读取数据的时候有可能没有读取完全，需要循环读取

    var tmp int
    for n < length {
        tmp, err = c.Read(buf[n:])
        if err != nil {
            break
        }
        n += tmp
    }

soulbird commented 1 year ago

func (r *Request) askExtraInfo(builder *flatbuffers.Builder,
  infoType ei.Info, info flatbuffers.UOffsetT) ([]byte, error) { 
       ......
  ty := header[0]
  header[0] = 0
  length := binary.BigEndian.Uint32(header)

  log.Infof("receive rpc type: %d data length: %d", ty, length)

  buf := make([]byte, length)
  n, err = c.Read(buf)
  if util.ReadErr(n, err, int(length)) {
      return nil, common.ErrConnClosed
  }
        ....
}

这里从conn读取数据的时候有可能没有读取完全，需要循环读取

  var tmp int
  for n < length {
      tmp, err = c.Read(buf[n:])
      if err != nil {
          break
      }
      n += tmp
  }

This is read in a loop.

for n < length

soulbird commented 1 year ago

@zdzh Looks like it's buggy, would you be interested in submiting a PR to fix it?

An-DJ commented 1 year ago

@spacewander Please assign to me.

An-DJ commented 1 year ago

@zdzh This issue has been solved by #124 .

apache / apisix-go-plugin-runner