search

ruby-protobuf / protobuf

A pure ruby implementation of Google's Protocol Buffers
https://github.com/ruby-protobuf
MIT License
463 stars 101 forks source link

encode straight to stream without intermediary copies #362

Closed abrandoned closed 7 years ago

abrandoned commented 7 years ago

Start reducing the amount of copying we do during encoding to a stream (which is the path taken because encode calls encode_to with a StringIO)

on MRI this doesn't make a significant difference

BEFORE

[4] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)> Benchmark.measure do
[4] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   10_000.times do  
[4] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*     proto.encode
[4] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   end  
[4] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)* end  
=> #<Benchmark::Tms:0x0000000116e2b0 @cstime=0.0, @cutime=0.0, @label="", @real=0.13301530084572732, @stime=0.0, @total=0.1299999999999999, @utime=0.1299999999999999>

AFTER

[4] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)> Benchmark.measure do
[4] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   10_000.times do  
[4] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*     proto.encode
[4] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   end  
[4] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)* end  
=> #<Benchmark::Tms:0x000000044deca8 @cstime=0.0, @cutime=0.0, @label="", @real=0.10333655099384487, @stime=0.0, @total=0.10000000000000009, @utime=0.10000000000000009>

But on JRuby the difference is more pronounced in the total and utime

BEFORE

[9] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)> Benchmark.measure do
[9] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   10_000.times do
[9] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*     proto.encode
[9] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   end  
[9] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)* end  
=> #<Benchmark::Tms:0x5cc87de4 @cstime=0.0, @cutime=0.0, @label="", @real=0.11006945697590709, @stime=0.0, @total=0.20000000000000284, @utime=0.20000000000000284>
[10] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)> Benchmark.measure do
[10] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   10_000.times do
[10] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*     proto.encode
[10] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   end  
[10] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)* end  
=> #<Benchmark::Tms:0x72a61e61 @cstime=0.0, @cutime=0.0, @label="", @real=0.10494967713020742, @stime=0.0, @total=0.3100000000000023, @utime=0.3100000000000023>
[11] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)> Benchmark.measure do
[11] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   10_000.times do
[11] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*     proto.encode
[11] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   end  
[11] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)* end  
=> #<Benchmark::Tms:0x5cf1bbd3 @cstime=0.0, @cutime=0.0, @label="", @real=0.1023686439730227, @stime=0.0, @total=0.28000000000000114, @utime=0.28000000000000114>

AFTER

[7] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)> Benchmark.measure do
[7] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   10_000.times do
[7] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*     proto.encode
[7] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   end  
[7] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)* end  
=> #<Benchmark::Tms:0x66859ea9 @cstime=0.0, @cutime=0.0, @label="", @real=0.09669065894559026, @stime=0.0, @total=0.14000000000000057, @utime=0.14000000000000057>
[8] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)> Benchmark.measure do
[8] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   10_000.times do
[8] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*     proto.encode
[8] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   end  
[8] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)* end  
=> #<Benchmark::Tms:0x3a359f7c @cstime=0.0, @cutime=0.0, @label="",@real=0.09078302001580596, @stime=0.010000000000000009, @total=0.13000000000000456, @utime=0.12000000000000455>
[9] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)> Benchmark.measure do
[9] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   10_000.times do
[9] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*     proto.encode
[9] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)*   end  
[9] pry(#<RSpec::ExampleGroups::ProtobufFieldStringField::Encode>)* end  
=> #<Benchmark::Tms:0x17063c32 @cstime=0.0, @cutime=0.0, @label="", @real=0.09246275201439857, @stime=0.0, @total=0.13000000000000256, @utime=0.13000000000000256>

@film42 @mmmries