zeromicro / go-zero

A cloud-native Go microservices framework with cli tool for productivity.
https://go-zero.dev
MIT License
29.44k stars 3.98k forks source link

About the bug of [func Parse(r *http.Request, v interface{}) error] #2633

Open george012 opened 2 years ago

george012 commented 2 years ago
func Parse(r *http.Request, v interface{}) error {
    if err := ParsePath(r, v); err != nil {
        return err
    }

    if err := ParseForm(r, v); err != nil {
        return err
    }

    if err := ParseHeaders(r, v); err != nil {
        return err
    }

    return ParseJsonBody(r, v)
}

上述方法中 追溯 [ return ParseJsonBody(r, v) ] 方法进入 [ rest.httpx.requests.go ] 中 第[85]行使用的是 [ reader := io.LimitReader(r.Body, maxBodyLen) ] 会在CGO 混编工程、或者使用unsafe.Pointer() 方法使用过后 报 [ string: `, error:EOF` ] 错误

尝试修改[reader := io.LimitReader(r.Body, maxBodyLen)] 为 [reader, := io.ReadAll(r.Body) ] 后错误解决!! 尝试修改[reader := io.LimitReader(r.Body, maxBodyLen)] 为 [reader, := io.ReadAll(r.Body) ] 后错误解决!! 尝试修改[reader := io.LimitReader(r.Body, maxBodyLen)] 为 [reader,_ := io.ReadAll(r.Body) ] 后错误解决!!

wsx864321 commented 2 years ago

Please provide scenarios that can be reproduced

george012 commented 2 years ago

①: Use [global middleware], use CGO library to operate data image ②: Data parsing error in [hadler] subsequent processing image Error message here: "string: `, error:EOF`" image

Issues-translate-bot commented 2 years ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


①: Use [global middleware], use CGO library to operate data image ②: Data parsing error in [hadler] subsequent processing image Error message here: "string: `, error:EOF`" image

george012 commented 2 years ago

msg

browningweb commented 1 year ago

Same question about this error. Any update?

browningweb commented 1 year ago

I have solved this question. Idea from https://stackoverflow.com/questions/23070876/reading-body-of-http-request-without-modifying-request-state

Here is the code.

func (rp *RequestParse) Parse(r *http.Request) (map[string]interface{}, error) {

    vars := pathvar.Vars(r)
    params, err := httpx.GetFormValues(r)
    if err != nil {
        return nil, err
    }

    for k, v := range vars {
        params[k] = v
    }

    body, ok := getBody(r)
    if !ok {
        return params, nil
    }

    m := make(map[string]interface{})
    if err := json.NewDecoder(body).Decode(&m); err != nil {
        return nil, err
    }

    for k, v := range params {
        m[k] = v
    }

    return m, nil
}

func getBody(r *http.Request) (io.Reader, bool) {
    if r.Body == nil {
        return nil, false
    }

    if r.ContentLength == 0 {
        return nil, false
    }

    buf, _ := io.ReadAll(r.Body)

    rdr1 := io.NopCloser(bytes.NewBuffer(buf))
    rdr2 := io.NopCloser(bytes.NewBuffer(buf))

    r.Body = rdr1

    return rdr2, true
}
wsx864321 commented 1 year ago

I have solved this question. Idea from https://stackoverflow.com/questions/23070876/reading-body-of-http-request-without-modifying-request-state

Here is the code.

func (rp *RequestParse) Parse(r *http.Request) (map[string]interface{}, error) {

  vars := pathvar.Vars(r)
  params, err := httpx.GetFormValues(r)
  if err != nil {
      return nil, err
  }

  for k, v := range vars {
      params[k] = v
  }

  body, ok := getBody(r)
  if !ok {
      return params, nil
  }

  m := make(map[string]interface{})
  if err := json.NewDecoder(body).Decode(&m); err != nil {
      return nil, err
  }

  for k, v := range params {
      m[k] = v
  }

  return m, nil
}

func getBody(r *http.Request) (io.Reader, bool) {
  if r.Body == nil {
      return nil, false
  }

  if r.ContentLength == 0 {
      return nil, false
  }

  buf, _ := io.ReadAll(r.Body)

  rdr1 := io.NopCloser(bytes.NewBuffer(buf))
  rdr2 := io.NopCloser(bytes.NewBuffer(buf))

  r.Body = rdr1

  return rdr2, true
}

This case is because you have read the body, actually, it is not a bug.

yiGmMk commented 1 year ago

同样是httpx.Parse报错"EOF" ` // go-zero版本 github.com/zeromicro/go-zero v1.5.5

// server配置了MaxBytes: 536870912 Name: img Host: 0.0.0.0 Port: 8888 MaxBytes: 536870912 Timeout: 600000

// api配置了maxBytes @server ( prefix: /v1 timeout: 300s // 对当前语法块下的所有路由添加请求体大小控制,512MB,单位为 byte maxBytes: 536870912 ) ` post请求,body数据超过8MB(8388608)就会报错,调试发现是这里报错 (调试时请求参数,ContentLength=11042730) image

把body的ContentLength降到8MB以下能正常请求

是bug吗?

@kevwan

Issues-translate-bot commented 1 year ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


The same httpx.Parse error "EOF"

github-actions[bot] commented 2 months ago

This issue is stale because it has been open for 30 days with no activity.