hashicorp / go-secure-stdlib

Mozilla Public License 2.0
64 stars 24 forks source link

Minimize allocations in strutil #19

Closed sethvargo closed 2 years ago

sethvargo commented 2 years ago

I noticed some pretty quick-and-easy wins for performance in the strutil package.

  1. Use struct{} instead of bool for map membership. The empty struct does not allocate and consumes no memory, so it's far more efficient than a boolean:

    package main
    
    import (
        "fmt"
        "unsafe"
    )
    
    func main() {
        x := [100]struct{}{}
        y := [100]bool{}
    
        fmt.Printf("%v\n", unsafe.Sizeof(x)) // 0
        fmt.Printf("%v\n", unsafe.Sizeof(y)) // 100
    }
  2. Pre-allocate slices and maps when we know the maximum possible size.

  3. Remove an unnecessary test dependency and rename the tests to match Go's expected test format name.

Benchmarks

Before:

BenchmarkRemoveDuplicates-16                   2     756790682 ns/op    106821536 B/op     38192 allocs/op
BenchmarkRemoveDuplicatesStable-16             6     183990038 ns/op    58343824 B/op          4 allocs/op
BenchmarkEquivalentSlices-16                   1    1666107197 ns/op    358676792 B/op     76680 allocs/op

After:

BenchmarkRemoveDuplicates-16                   3     477817660 ns/op    56117237 B/op          7 allocs/op
BenchmarkRemoveDuplicatesStable-16             6     200917214 ns/op    56115224 B/op          3 allocs/op
BenchmarkEquivalentSlices-16                   1    1009316412 ns/op    112230496 B/op         8 allocs/op
sethvargo commented 2 years ago

Pokey @jefferai @calvn

jefferai commented 2 years ago

I'm on vacation :-)

jefferai commented 2 years ago

Thanks!

sethvargo commented 2 years ago

Thank you 😄