search

apache / apisix-go-plugin-runner

Go Plugin Runner for APISIX
https://apisix.apache.org/
Apache License 2.0
167 stars 69 forks source link

bug: rpc error: code = Internal desc = server closed the stream without sending trailers #106

Closed gitcome closed 1 year ago

gitcome commented 1 year ago

Current Behavior

apisix: apache/apisix:2.15.0-alpine plugin-runner: apache/apisix-go-plugin-runner v0.4.0

run apisix-go-plugin-runner ,in func RequestFilter, WriteHeader then return, grpc client will report an error(code = Internal desc = server closed the stream without sending trailers)

I studied for a day and didn't know how to solve the problem. can anybody help solve the problem? thanks a lot!

Expected Behavior

grpc client not report error,and can get trailers

Error Logs

rpc error: code = Internal desc = server closed the stream without sending trailers

Steps to Reproduce

apisix: apache/apisix:2.15.0-alpine plugin-runner: apache/apisix-go-plugin-runner v0.4.0

  1. modify code in apisix-go-plugin-runner-master\cmd\go-runner\plugins\say.go, as follows:
func (p *Say) RequestFilter(conf interface{}, w http.ResponseWriter, r pkgHTTP.Request) {
    w.Header().Set("Content-Type", r.Header().Get("Content-Type"))
    num := rand.Int()
    if num%2 == 1 {
        w.WriteHeader(http.StatusOK)
        w.Header().Set("mykey", "test")
        w.Header().Set("Grpc-Status", "0")
        w.Header().Set("Grpc-Message", "ok")
        w.Header().Set("Grpc-Status-Details-Bin", "")
        return
    }
}

run a grpc server in 192.168.56.19:8301

  1. then run apisix, config upstream in APISIX Dashboard，as follows:

    {
"nodes": [
{
  "host": "192.168.56.19",
  "port": 8301,
  "weight": 1
}
],
"timeout": {
"connect": 6,
"send": 6,
"read": 6
},
"type": "roundrobin",
"scheme": "grpc",
"pass_host": "pass",
"name": "testa",
"keepalive_pool": {
"idle_timeout": 60,
"requests": 1000,
"size": 320
}
}

  2. then add a route config, as follows:

    {
"uri": "/routeguide.RouteGuide/GetFeature",
"name": "testa1",
"methods": [
"GET",
"POST",
"PUT",
"DELETE",
"PATCH",
"HEAD",
"OPTIONS",
"CONNECT",
"TRACE"
],
"plugins": {
"ext-plugin-post-req": {
  "conf": [
    {
      "name": "say",
      "value": "{\"body\":\"123\"}"
    }
  ],
  "disable": false
}
},
"upstream_id": "425744995673178819",
"status": 1
}

  3. then use client grpc call apisix, but have an error (rpc error: code = Internal desc = server closed the stream without sending trailers)，as follows:

2022/09/21 19:45:47.860 [I] [main.go:59]  header: map[content-type:[application/grpc] date:[Wed, 21 Sep 2022 11:45:47 GMT] mykey:[test] server:[APISIX/2.15.0]]
2022/09/21 19:45:47.860 [I] [main.go:60]  trailer: map[]
2022/09/21 19:45:47.861 [E] [main.go:62]  rpc error: code = Internal desc = server closed the stream without sending trailers

Environment

os: win10 apisix: apache/apisix:2.15.0-alpine (use apisix-docker\example\docker-compose.yml docker-compose up) plugin-runner: apache/apisix-go-plugin-runner v0.4.0

soulbird commented 1 year ago

If you want to use the grpc client to access APISIX, you need to enable the grpc-transcode plugin.

"scheme": "grpc" in upstream configures the protocol for accessing upstream to be grpc.

gitcome commented 1 year ago

If you want to use the grpc client to access APISIX, you need to enable the grpc-transcode plugin.

"scheme": "grpc" in upstream configures the protocol for accessing upstream to be grpc.

I've read this grpc-transcode.md, but it doesn't seem to be relevant to this problem. Can you contact me via WeChat 'program_no_error'? thanks a lot!

gitcome commented 1 year ago

If you want to use the grpc client to access APISIX, you need to enable the grpc-transcode plugin.

"scheme": "grpc" in upstream configures the protocol for accessing upstream to be grpc.

my call chain like this:

                                                           1.[RequestFilter return] --?--> GrpcClient (report error)
GrpcClient --grpc--> Apisix ----> Go-plugin-runner(ext-plugin-post-req)  
                                                           2.[RequestFilter continue] ----> GrpcServer(upstream) ----> Apisix ----> GrpcClient(ok)

in Go-plugin-runner, func RequestFilter like this:

func (p *Say) RequestFilter(conf interface{}, w http.ResponseWriter, r pkgHTTP.Request) {
    w.Header().Set("Content-Type", r.Header().Get("Content-Type"))
    num := rand.Int()
    if num%2 == 1 {
        w.WriteHeader(http.StatusOK)
        w.Header().Set("mykey", "test")
        w.Header().Set("Grpc-Status", "0")
        w.Header().Set("Grpc-Message", "ok")
        w.Header().Set("Grpc-Status-Details-Bin", "")
        return   // here retrun to GrpcClient, see above chain 1, client report "server closed the stream without sending trailers"
    }
        return // here continue to GrpcServer(upstream), then return to GrpcClient, see above chain 2, ok
}

above not use grpc-transcode，grpc-transcode not suitable for our scene。Please have a look and help， thanks a lot!

soulbird commented 1 year ago

Regarding this step GrpcClient --grpc--> Apisix, how did you do it? What tool or command was used ?

gitcome commented 1 year ago

Regarding this step GrpcClient --grpc--> Apisix, how did you do it? What tool or command was used ?

client use grpc protocol request Apisix. code as follows:

    conn, err := grpc.Dial("192.168.0.1:9080", grpc.WithTransportCredentials(insecure.NewCredentials()))
    if nil != err {
        return
    }
    defer func() {
        conn.Close()
    }()
    client := routeguide.NewRouteGuideClient(conn)
    md := metadata.MD{}
    md.Set("header1", "test")
    ctx = metadata.NewOutgoingContext(ctx, md)
    var rspheader metadata.MD
    var rsptrailer metadata.MD
    req := routeguide.Point{Latitude: 99, Longitude: 88}
    rsp, err := cli.GetFeature(ctx , &req, grpc.Header(&rspheader), grpc.Trailer(&rsptrailer))
    if err != nil {
        logs.Error(err)      //here report error "server closed the stream without sending trailers"
        return
    }
    logs.Info("rsp:", rsp.String())
        logs.Info("rspheader:", rspheader)
        logs.Info("rsptrailer:", rsptrailer)
soulbird commented 1 year ago

It seems that it is the same issue as https://github.com/apache/apisix/issues/7973. Please do not submit the same issue in the two projects, it will be inconvenient to track, and information will be lost.

gitcome commented 1 year ago

ok, sorry, i close this