apple / swift-collections

Commonly used data structures for Swift
Apache License 2.0
3.55k stars 270 forks source link

[Benchmarks] Add Collection Equality Benchmarks #351

Closed vanvoorden closed 3 months ago

vanvoorden commented 4 months ago

Background

We have benchmarks defined to measure the performance of OrderedSet and OrderedDictionary equality to confirm that equality of shared instances perform efficiently (in O(1) time). We would like to add benchmarks for TreeSet and TreeDictionary to confirm the same behavior. We can also go ahead and add those same benchmarks for the system Set and Dictionary implementations.

Changes

We only make changes to benchmarks. This diff contains no implementation changes.

In addition to the new benchmarks, we can also make small changes to previous benchmarks to make them more consistent (we migrate same instance to shared and different instance to unique.

Benchmarks

swift run -c release benchmark run results.json --cycles 5 --tasks \
"Set<Int> equality, unique" \
"OrderedSet<Int> equality, unique" \
"OrderedSet<Int>.SubSequence equality, unique" \
"TreeSet<Int> equality, unique" \
"Set<Int> equality, shared" \
"OrderedSet<Int> equality, shared" \
"OrderedSet<Int>.SubSequence equality, shared" \
"TreeSet<Int> equality, shared"

swift run -c release benchmark render results.json set-unique.png --amortized false --tasks \
"Set<Int> equality, unique" \
"OrderedSet<Int> equality, unique" \
"OrderedSet<Int>.SubSequence equality, unique" \
"TreeSet<Int> equality, unique"

swift run -c release benchmark render results.json set-shared.png --amortized false --tasks \
"Set<Int> equality, shared" \
"OrderedSet<Int> equality, shared" \
"OrderedSet<Int>.SubSequence equality, shared" \
"TreeSet<Int> equality, shared"

swift run -c release benchmark run results.json --cycles 5 --tasks \
"Dictionary<Int, Int> equality, unique" \
"OrderedDictionary<Int, Int> equality, unique" \
"OrderedDictionary<Int, Int>.Values equality, unique" \
"Dictionary<Int, Int> equality, shared" \
"OrderedDictionary<Int, Int> equality, shared" \
"OrderedDictionary<Int, Int>.Values equality, shared" \

swift run -c release benchmark render results.json dictionary-unique.png --amortized false --tasks \
"Dictionary<Int, Int> equality, unique" \
"OrderedDictionary<Int, Int> equality, unique" \
"OrderedDictionary<Int, Int>.Values equality, unique" \

swift run -c release benchmark render results.json dictionary-shared.png --amortized false --tasks \
"Dictionary<Int, Int> equality, shared" \
"OrderedDictionary<Int, Int> equality, shared" \
"OrderedDictionary<Int, Int>.Values equality, shared" \
dictionary-shared dictionary-unique set-shared set-unique

Checklist

lorentey commented 3 months ago

@swift-ci test

lorentey commented 3 months ago

CI won't look at Benchmarks/, so I'll just merge this as is.