Closed anhnmt closed 8 months ago
Hey @anhnmt thanks for trying out the library! For dynamic protobuf files you can create the service with vanguard.NewServiceWithSchema
. See here for details on how to dynamically load protobufs.
To clarify is it a gRPC server on localhost:8080? To create a reverse proxy you could setup the proxy server as:
var schema protoreflect.ServiceDescriptor // Loaded dynamically
remote, _ := url.Parse("http://localhost:8080")
proxy := httputil.NewSingleHostReverseProxy(remote)
transcoder := vanguard.NewTranscoder(
[]*vanguard.Service{
vanguard.NewServiceWithSchema(schema, proxy),
},
)
@anhnmt, hi! Glad you are finding this repo valuable!
For a little more info on obtaining descriptors, to use with vanguard.NewServiceWithSchema
, you'll do something like this:
google.protobuf.FileDescriptorSet
message. You'll unmarshal the file contents into a new message of that type (*descriptorpb.FileDescriptorSet
in the Go runtime). From there, you can create a *protoregistry.Files
using protodesc.NewFiles
and then create a type resolver using dynamicpb.NewTypes
.
files.FindDescriptorByName
. You can then type-assert the result to a protoreflect.ServiceDescriptor
and provide to vanguard.NewServiceWithSchema
.
vanguard.WithTypeResolver(types)
. This allows for the same schema file to be consulted when marshaling and un-marshaling extensions and google.protobuf.Any
messages.@emcfarlane I use httputil.NewSingleHostReverseProxy but it seems it only reverses HTTP proxies, gRPC does not. Do you have any method to reverse gRPC proxy?
@anhnmt, gRPC uses HTTP under the hood. If you are having an issue, it is likely due to use of HTTP 1.1, whereas gRPC requires HTTP/2. If you are not using TLS (where the HTTP/2 protocol can be negotiated during the TLS handshake), then you need to be using "HTTP/2 over clear text", also called "H2C". There is more detail in the Connect docs, since this is also necessary to use the gRPC protocol with the connect-go module: https://connectrpc.com/docs/go/deployment/#h2c
In particular, you'd need to wrap your reverse proxy handler using h2c.NewHandler
as in the code snippet in that link, if your proxy server accepts gRPC requests without TLS. And you need to set the Transport
field of the httputil.ReverseProxy
to a transport implementation supports H2C. The link above shows an example of creating an http2.Transport
that works over plaintext.
@jhump, I used h2c but it seems I made a mistake in some step so I still can't connect to gRPC Here is my source code: https://github.com/anhnmt/gprc-dynamic-proto/blob/main/cmd/dynamic/main.go
Hey @anhnmt, I think the reverse proxy should be created with a http2 transport here:
proxy := httputil.NewSingleHostReverseProxy(target)
proxy.Transport = &http2.Transport{AllowHTTP: true}
h2cHandler := h2c.NewHandler(proxy, &http2.Server{})
Let me know if this works for you. You may need to configure the transport for your server configuration.
@emcfarlane
proxy.Transport = &http2.Transport{AllowHTTP: true}
I also tried your method but encountered some other problems, for example:
proxy := httputil.NewSingleHostReverseProxy(target)
+ proxy.Transport = &http2.Transport{
+ AllowHTTP: true,
+ DialTLS: func(network, addr string, _ *tls.Config) (net.Conn, error) {
+ // If you're also using this client for non-h2c traffic, you may want
+ // to delegate to tls.Dial if the network isn't TCP or the addr isn't
+ // in an allowlist.
+ return net.Dial(network, addr)
+ },
+ }
h2cHandler := h2c.NewHandler(proxy, &http2.Server{})
Tested with curl localhost:8000/v1/users/1
and looks like it works!
@emcfarlane Thank you very much 😁 I also found a similar way here: https://github.com/connectrpc/vanguard-go/blob/d7faf709c8be3a7e94649ac858e83107fe2d88e6/internal/examples/pets/cmd/pets-fe/main.go#L66-L71
Also, can you give me some feedback on optimizing this part?
import (
"bytes"
"fmt"
"io"
"net/http"
"net/http/httputil"
"net/url"
"os"
"path/filepath"
"time"
"connectrpc.com/vanguard"
"github.com/jhump/protoreflect/desc/protoparse"
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
"golang.org/x/net/http2"
"golang.org/x/net/http2/h2c"
"google.golang.org/protobuf/reflect/protodesc"
"google.golang.org/protobuf/types/descriptorpb"
"google.golang.org/protobuf/types/dynamicpb"
)
googleapis := []string{
"google/api/annotations.proto",
"google/api/http.proto",
"google/protobuf/descriptor.proto",
}
files := append(googleapis, "user/v1/user.proto")
p := protoparse.Parser{
ImportPaths: []string{
"proto",
"googleapis",
},
Accessor: func(filename string) (io.ReadCloser, error) {
return ReadFileContent(filename)
},
}
fds, err := p.ParseFiles(files...)
if err != nil {
log.Err(err).Msg("could not parse given files")
return
}
fileDescriptors := make([]*descriptorpb.FileDescriptorProto, 0)
for _, fd := range fds {
fileDescriptors = append(fileDescriptors, fd.AsFileDescriptorProto())
}
newFiles, err := protodesc.NewFiles(&descriptorpb.FileDescriptorSet{
File: fileDescriptors,
})
if err != nil {
log.Err(err).Msg("could not parse given files")
return
}
types := dynamicpb.NewTypes(newFiles)
name, err := newFiles.FindDescriptorByName("user.v1.UserService")
if err != nil {
return
}
serviceDesc := name.ParentFile().Services().ByName("UserService")
The use of protoparse
is really interesting! If you want to parse the descriptors up front you could use buf's images. See the docs here. Then build the image up front and include the filedescriptor sets dynamically:buf build -o users.binpb
. @jhump will have better advice!
A bug I noticed in the proto. The option in users.proto
:
option (google.api.http) = {get: "/v1/users/{page=**}"};
Should be {page=*}
(or just {page}
) as **
will never be able to be unmarshalled into the page int32 field. There does look like an issue with stacking multiple Transcoder
instances and error handling.
curl localhost:8080/v1/users/1/123
{"code":3,"message":"invalid parameter \"page\" invalid character '/' after top-level value","details":[]}
curl localhost:8000/v1/users/1/123
{"code":2, "message":"response uses incorrect codec: expecting \"json\" but instead got \"?\"", "details":[]}
I saw the use of protocompile
in the commented out code. Was there a reason you're using the older protoparse
instead? The result of protocompile
is a slice of protoreflect.FileDescriptor
instances, but the slice is a named type (linker.Files
) that also has a method AsResolver()
, so you can use that to look up descriptors by name. That way no conversion or registry construction are necessary.
In the existing code, which does convert descriptors and construct a registry (using protodesc.NewFiles
), one strange thing here is that when converting from *desc.FileDescriptor
-> protoreflect.FileDescriptor
, the code re-builds the protoreflect.FilDescriptor
from the underlying proto. But that isn't necessary if you are using v1.15+ of github.com/jhump/protoreflect/desc/protoparse
. You can instead just use fileDesc.UnwrapFile()
. So creating a resolver with all of them could instead look like so:
resolver := &protoregistry.Files{}
for _, fileDesc := range parsedFiles {
if err := resolver.RegisterFile(fileDesc.UnwrapFile()); err != nil {
return err
}
}
(Having said that, using protocompile
would be even better.)
As @emcfarlane said, you could use an image or file descriptor set file, which may be simpler to distribute to production containers than all of the source needed to compile the schema. And if you push the code to the BSR, then you could instead download the schema via a reflection endpoint. In fact, there's a Go library that includes the ability to watch a schema in the BSR, downloading a new version when one becomes available (and, under the hood, it uses that reflection endpoint).
@jhump, I used protocompile
and got the following error, while github.com/jhump/protoreflect
did not get this case
My source code: https://github.com/anhnmt/gprc-dynamic-proto/blob/fc77aa12da23fbaaa171e97b4814c8185ba5ab05/cmd/protocompile/main.go
panic: invalid type: got *dynamicpb.Message, want *annotations.HttpRule
goroutine 1 [running]:
google.golang.org/protobuf/internal/impl.(*messageConverter).GoValueOf(0xc00050bb40, {{}, 0xdef640?, 0xc0004a9bc0?, 0xedee28?})
C:/Users/ANHNMT/go/pkg/mod/google.golang.org/protobuf@v1.31.0/internal/impl/convert.go:457 +0x408
google.golang.org/protobuf/internal/impl.(*ExtensionInfo).InterfaceOf(0x12a1ee0?, {{}, 0xdef640?, 0xc0004a9bc0?, 0x12a1f00?})
C:/Users/ANHNMT/go/pkg/mod/google.golang.org/protobuf@v1.31.0/internal/impl/extension.go:102 +0x5a
google.golang.org/protobuf/proto.GetExtension({0xed7140?, 0xc00009a9c0?}, {0xedee28, 0x12a1ee0})
C:/Users/ANHNMT/go/pkg/mod/google.golang.org/protobuf@v1.31.0/proto/extension.go:45 +0x9b
connectrpc.com/vanguard.getHTTPRuleExtension({0xee0e88?, 0xc0002022a0?})
C:/Users/ANHNMT/go/pkg/mod/connectrpc.com/vanguard@v0.1.0/protocol_http.go:342 +0x66
connectrpc.com/vanguard.(*Transcoder).registerMethod(0xc000202480, {0xed6360?, 0xc00052a8d0}, {0xee0e88, 0xc0002022a0}, 0xc0004a9c40)
C:/Users/ANHNMT/go/pkg/mod/connectrpc.com/vanguard@v0.1.0/transcoder.go:204 +0x2e5
connectrpc.com/vanguard.(*Transcoder).registerService(0xc000202480, 0xc0003b5ee8, {{0xedc538, 0xc0000403f0}, 0xc000539f20, 0xc000539ec0, 0xc000539ef0, {0xe09b31, 0x5}, 0xffffffff, ...})
C:/Users/ANHNMT/go/pkg/mod/connectrpc.com/vanguard@v0.1.0/transcoder.go:127 +0x6f8
connectrpc.com/vanguard.NewTranscoder({0xc0003b5e58, 0x1, 0xb?}, {0x0, 0x0, 0x6?})
C:/Users/ANHNMT/go/pkg/mod/connectrpc.com/vanguard@v0.1.0/vanguard.go:131 +0x628
main.main()
C:/Golang/gprc-dynamic-proto/cmd/dynamic/main.go:149 +0x6bc
Process finished with the exit code 2
@anhnmt, ah, right. Sorry, I forgot about the HTTP annotations. It is indeed expected that they are dynamic extensions from the compiler (since the compiler's version of the message definition could differ from the version linked into the calling program).
Seems like a good addition to protocompile would be a helper to address that, or maybe even a compiler option so that you don't have to do anything as a post-process. The helper or option would basically do the same thing that protoparse does, which is to re-parse custom options using protoregistry.GlobalTypes
as the resolver.
I have a proto file and gprc client address localhost:8080
How can I connect to gprc client using the above proto file without having to generate .pb.go or .connect.go files?
This mechanism of action may look like this: https://docs.konghq.com/hub/kong-inc/grpc-gateway/how-to/
Also I quite enjoy using this library, thank you for creating this wonderful library 😁