imiric
commented
3 years ago Hi @thecodejunkie, thanks for reporting this and for the comprehensive code to reproduce it.
You're right, this is a surprising difference and we'd also expect better performance from the gRPC test, especially since you're reusing the TCP connection.
I took a look at it and didn't find anything immediately obvious. Some notes:
-
There is some JSON marshaling overhead in the gRPC implementation, but even after removing it it accounted for about ~10% of the difference, so that's not the main issue.
-
The Fiber framework uses fasthttp which is a bit faster than stdlib's
net/httpand would show a bigger difference, but even when using anet/http-based server the difference remains around 40%. I similarly tried removingcmuxand theContent-Typematching from the example, but didn't notice any improvements. -
From profiling the gRPC test most of the time is spent on GC, goja and
grpc.invoke, as expected, and I'm not clear if any optimizations can be done there, though it might be worth looking into.
So we would need more time to dig into this and determine the root cause. Please give us a few days to investigate and discuss this internally.
thecodejunkie
LaserPhaser
na--
codebien
kiwi-waquar
joanlopez
Hi,
I was chatting with @simskij, on the Gophers Slack about some performance observations I was having when using k6 to compare gRPC and HTTP endpoints (in the same service), and he suggested I post here as well to get some insights from the performance wizards :)
Both endpoints have a similar setup (for testing purpose) and for gRPC I get about 32k req/s and for HTTP I am seeing about 55k req/s. I was expecting gRPC to be a bit higher, but I might be wrong?
I run my service and k6 tests on the same machine, a 2.2Ghz 6 core i7 MBP with 32GB RAM. Below are the test results
Environment
Expected Behavior
I was expecting gRPC to be on par (or even faster) then my HTTP endpoint? @simskij was seeing similar results on his MBP, but wasn't 100% sure of the difference in results (i.e if it was to be expected and why, or if something other was at play)
Actual Behavior
gRPC performs at (almost) 50% of the HTTP endpoint
Steps to Reproduce the Problem
I've included all the code to reproduce this
main.go
```go package main import ( "context" "encoding/json" "google.golang.org/grpc" "log" "net" "github.com/gofiber/fiber/v2" "github.com/soheilhy/cmux" "go-grpc-http-muxer.com/chat" ) func main() { l, err := net.Listen("tcp", ":3000") if err != nil { log.Panic(err) } m := cmux.New(l) // Create a grpc listener first grpcListener := m.MatchWithWriters(cmux.HTTP2MatchHeaderFieldSendSettings("content-type", "application/grpc")) // All the rest is assumed to be HTTP httpListener := m.Match(cmux.Any()) go serveHTTP(httpListener) go serveGRPC(grpcListener) _ = m.Serve() } type ChatServer struct { } func (c *ChatServer) SayHello(context.Context, *chat.Message) (*chat.Message, error) { m := &chat.Message{ Body: "Hello, World 👋!", } return m, nil } func serveGRPC(l net.Listener) { s := &ChatServer{} grpcServer := grpc.NewServer() chat.RegisterChatServiceServer(grpcServer, s) if err := grpcServer.Serve(l); err != nil { log.Fatalf("failed to serve: %s", err) } } func serveHTTP(l net.Listener) { app := fiber.New() app.Get("/", func(c *fiber.Ctx) error { m := &chat.Message{ Body: "Hello, World 👋!", } b, _ := json.Marshal(m) return c.Send(b) }) app.Listener(l) } ```
chat.proto
```proto syntax = "proto3"; package go.grpc.http.muxer.com.chat.v1; option go_package = ".;chat"; message Message { string body = 1; } service ChatService { rpc SayHello(Message) returns (Message) {} } ```
chat.pb.go
```go // Code generated by protoc-gen-go. DO NOT EDIT. // versions: // protoc-gen-go v1.25.0-devel // protoc v3.14.0 // source: chat.proto package chat import ( context "context" grpc "google.golang.org/grpc" codes "google.golang.org/grpc/codes" status "google.golang.org/grpc/status" protoreflect "google.golang.org/protobuf/reflect/protoreflect" protoimpl "google.golang.org/protobuf/runtime/protoimpl" reflect "reflect" sync "sync" ) const ( // Verify that this generated code is sufficiently up-to-date. _ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion) // Verify that runtime/protoimpl is sufficiently up-to-date. _ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20) ) type Message struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields Body string `protobuf:"bytes,1,opt,name=body,proto3" json:"body,omitempty"` } func (x *Message) Reset() { *x = Message{} if protoimpl.UnsafeEnabled { mi := &file_chat_proto_msgTypes[0] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } } func (x *Message) String() string { return protoimpl.X.MessageStringOf(x) } func (*Message) ProtoMessage() {} func (x *Message) ProtoReflect() protoreflect.Message { mi := &file_chat_proto_msgTypes[0] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) } return ms } return mi.MessageOf(x) } // Deprecated: Use Message.ProtoReflect.Descriptor instead. func (*Message) Descriptor() ([]byte, []int) { return file_chat_proto_rawDescGZIP(), []int{0} } func (x *Message) GetBody() string { if x != nil { return x.Body } return "" } var File_chat_proto protoreflect.FileDescriptor var file_chat_proto_rawDesc = []byte{ 0x0a, 0x0a, 0x63, 0x68, 0x61, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x1e, 0x67, 0x6f, 0x2e, 0x67, 0x72, 0x70, 0x63, 0x2e, 0x68, 0x74, 0x74, 0x70, 0x2e, 0x6d, 0x75, 0x78, 0x65, 0x72, 0x2e, 0x63, 0x6f, 0x6d, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x2e, 0x76, 0x31, 0x22, 0x1d, 0x0a, 0x07, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x32, 0x6d, 0x0a, 0x0b, 0x43, 0x68, 0x61, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x5e, 0x0a, 0x08, 0x53, 0x61, 0x79, 0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x12, 0x27, 0x2e, 0x67, 0x6f, 0x2e, 0x67, 0x72, 0x70, 0x63, 0x2e, 0x68, 0x74, 0x74, 0x70, 0x2e, 0x6d, 0x75, 0x78, 0x65, 0x72, 0x2e, 0x63, 0x6f, 0x6d, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x2e, 0x76, 0x31, 0x2e, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x1a, 0x27, 0x2e, 0x67, 0x6f, 0x2e, 0x67, 0x72, 0x70, 0x63, 0x2e, 0x68, 0x74, 0x74, 0x70, 0x2e, 0x6d, 0x75, 0x78, 0x65, 0x72, 0x2e, 0x63, 0x6f, 0x6d, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x2e, 0x76, 0x31, 0x2e, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x22, 0x00, 0x42, 0x08, 0x5a, 0x06, 0x2e, 0x3b, 0x63, 0x68, 0x61, 0x74, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, } var ( file_chat_proto_rawDescOnce sync.Once file_chat_proto_rawDescData = file_chat_proto_rawDesc ) func file_chat_proto_rawDescGZIP() []byte { file_chat_proto_rawDescOnce.Do(func() { file_chat_proto_rawDescData = protoimpl.X.CompressGZIP(file_chat_proto_rawDescData) }) return file_chat_proto_rawDescData } var file_chat_proto_msgTypes = make([]protoimpl.MessageInfo, 1) var file_chat_proto_goTypes = []interface{}{ (*Message)(nil), // 0: go.grpc.http.muxer.com.chat.v1.Message } var file_chat_proto_depIdxs = []int32{ 0, // 0: go.grpc.http.muxer.com.chat.v1.ChatService.SayHello:input_type -> go.grpc.http.muxer.com.chat.v1.Message 0, // 1: go.grpc.http.muxer.com.chat.v1.ChatService.SayHello:output_type -> go.grpc.http.muxer.com.chat.v1.Message 1, // [1:2] is the sub-list for method output_type 0, // [0:1] is the sub-list for method input_type 0, // [0:0] is the sub-list for extension type_name 0, // [0:0] is the sub-list for extension extendee 0, // [0:0] is the sub-list for field type_name } func init() { file_chat_proto_init() } func file_chat_proto_init() { if File_chat_proto != nil { return } if !protoimpl.UnsafeEnabled { file_chat_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Message); i { case 0: return &v.state case 1: return &v.sizeCache case 2: return &v.unknownFields default: return nil } } } type x struct{} out := protoimpl.TypeBuilder{ File: protoimpl.DescBuilder{ GoPackagePath: reflect.TypeOf(x{}).PkgPath(), RawDescriptor: file_chat_proto_rawDesc, NumEnums: 0, NumMessages: 1, NumExtensions: 0, NumServices: 1, }, GoTypes: file_chat_proto_goTypes, DependencyIndexes: file_chat_proto_depIdxs, MessageInfos: file_chat_proto_msgTypes, }.Build() File_chat_proto = out.File file_chat_proto_rawDesc = nil file_chat_proto_goTypes = nil file_chat_proto_depIdxs = nil } // Reference imports to suppress errors if they are not otherwise used. var _ context.Context var _ grpc.ClientConnInterface // This is a compile-time assertion to ensure that this generated file // is compatible with the grpc package it is being compiled against. const _ = grpc.SupportPackageIsVersion6 // ChatServiceClient is the client API for ChatService service. // // For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream. type ChatServiceClient interface { SayHello(ctx context.Context, in *Message, opts ...grpc.CallOption) (*Message, error) } type chatServiceClient struct { cc grpc.ClientConnInterface } func NewChatServiceClient(cc grpc.ClientConnInterface) ChatServiceClient { return &chatServiceClient{cc} } func (c *chatServiceClient) SayHello(ctx context.Context, in *Message, opts ...grpc.CallOption) (*Message, error) { out := new(Message) err := c.cc.Invoke(ctx, "/go.grpc.http.muxer.com.chat.v1.ChatService/SayHello", in, out, opts...) if err != nil { return nil, err } return out, nil } // ChatServiceServer is the server API for ChatService service. type ChatServiceServer interface { SayHello(context.Context, *Message) (*Message, error) } // UnimplementedChatServiceServer can be embedded to have forward compatible implementations. type UnimplementedChatServiceServer struct { } func (*UnimplementedChatServiceServer) SayHello(context.Context, *Message) (*Message, error) { return nil, status.Errorf(codes.Unimplemented, "method SayHello not implemented") } func RegisterChatServiceServer(s *grpc.Server, srv ChatServiceServer) { s.RegisterService(&_ChatService_serviceDesc, srv) } func _ChatService_SayHello_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(Message) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(ChatServiceServer).SayHello(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/go.grpc.http.muxer.com.chat.v1.ChatService/SayHello", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(ChatServiceServer).SayHello(ctx, req.(*Message)) } return interceptor(ctx, in, info, handler) } var _ChatService_serviceDesc = grpc.ServiceDesc{ ServiceName: "go.grpc.http.muxer.com.chat.v1.ChatService", HandlerType: (*ChatServiceServer)(nil), Methods: []grpc.MethodDesc{ { MethodName: "SayHello", Handler: _ChatService_SayHello_Handler, }, }, Streams: []grpc.StreamDesc{}, Metadata: "chat.proto", } ```
grpc.js
```js import grpc from 'k6/net/grpc'; import { check, sleep } from "k6"; export let options = { vus: 50, duration: '10s' } let client = new grpc.Client(); client.load([], "chat.proto") export default () => { if (__ITER == 0) { client.connect("127.0.0.1:3000", { plaintext: true }) } const response = client.invoke("go.grpc.http.muxer.com.chat.v1.ChatService/SayHello", { body: 'hi' }) check(response, { "status is OK": (r) => r && r.status === grpc.StatusOK }); } ```
http.js
```js import http from 'k6/http'; import { check } from "k6"; export let options = { vus: 50, duration: '10s' } export default () => { let res = http.get('http://127.0.0.1:3000/'); check(res, { 'is status 200': (r) => r.status === 200, }); } ```