tendermint / go-amino

Protobuf3 with Interface support - Designed for blockchains (deterministic, upgradeable, fast, and compact)
Other
260 stars 78 forks source link

Investigate amino performance/mem usage #254

Open liamsi opened 5 years ago

liamsi commented 5 years ago

While looking into adding amino to this list: https://github.com/alecthomas/go_serialization_benchmarks (see this branch: https://github.com/Liamsi/go_serialization_benchmarks/tree/add_amino), it seems like amino is quite slow compared to other similar libraries:

BenchmarkAminoMarshal-12                         300000              5492 ns/op            4138 B/op         72 allocs/op
BenchmarkAminoUnmarshal-12                       1000000              1081 ns/op             194 B/op          7 allocs/op
BenchmarkProtobufMarshal-12                      2000000               676 ns/op             200 B/op          7 allocs/op
BenchmarkProtobufUnmarshal-12                    3000000               593 ns/op             192 B/op         10 allocs/op
BenchmarkGoprotobufMarshal-12                    5000000               325 ns/op              96 B/op          2 allocs/op
BenchmarkGoprotobufUnmarshal-12                  3000000               498 ns/op             200 B/op         10 allocs/op
BenchmarkGogoprotobufMarshal-12                 20000000               119 ns/op              64 B/op          1 allocs/op
BenchmarkGogoprotobufUnmarshal-12               10000000               166 ns/op              96 B/op          3 allocs/op

Hopefully, we should be able to vastly improve this performance without completely reworking the structure.

Here is what the profiler tells us about memory/cpu while running above benchmarks for unmarshaling/marshaling:



marshal_cpu_profile


marshal_mem_profile


unmarshal_cpu_profile


unmarshal_mem_profile


Zaki suggested to add a hint in the readme about performance issue and in which cases users would might want to refrain from using amino.

rickyyangz commented 4 years ago

Two things are not fair for your amino case.

  1. Before marshalling, you need to reset timer or stop timer before generating testing data.

    func BenchmarkAminoMarshal(b *testing.B) {
        // b.StopTimer()
    data := generateAmino()
    b.ReportAllocs()
    b.StartTimer() 
    s := AminoSerializer{amino.NewCodec()}
    for i := 0; i < b.N; i++ {
            s.MustMarshalBinaryBare(data[rand.Intn(len(data))])
    }
    }
  2. the type of BirthDay is time.Time while in other cases it is int64

liamsi commented 4 years ago

good point @rickyyangz. Did you you re-benchmark with your suggested changes? I would assume the performance to still be much slower than generated protobuf.