betcode-org / betfair

betfairlightweight - Betfair API-NG python wrapper (with streaming)
MIT License
428 stars 147 forks source link

Improve performance of BaseStream.snap #533

Closed petedmarsh closed 1 year ago

petedmarsh commented 1 year ago

This speeds up snap by splitting it into two cases, where market_ids is None or not.

If market_ids are specified then we can avoid construcing a list of all of self._caches.values(). To do this the walrus operator is used. This means that Python >= 3.8 is required, but that is already the oldest tested Python version.

If market_ids is None then we can avoid the conditional on market_ids being None or not on each iteration.

Benchmark:

https://gist.github.com/petedmarsh/71337df6a09d47e5f55d2f0d60e15cf7

Results (Python 3.11.3):

                                                                Benchmarks, repeat=5, number=5
     ┌──────────────────────────────────────────────────────────────────┬─────────┬─────────┬─────────┬────────────────┬─────────────────┬────────────────┐
     │                                                        Benchmark │ Min     │ Max     │ Mean    │ Min (+)        │ Max (+)         │ Mean (+)       │
     ├──────────────────────────────────────────────────────────────────┼─────────┼─────────┼─────────┼────────────────┼─────────────────┼────────────────┤
     │ Snap Original vs New with specified market_ids and no contention │ 0.817   │ 0.830   │ 0.827   │ 0.226 (3.6x)   │ 0.231 (3.6x)    │ 0.228 (3.6x)   │
     │      Snap Original vs New with market_ids=None and no contention │ 0.360   │ 0.365   │ 0.363   │ 0.316 (1.1x)   │ 0.323 (1.1x)    │ 0.319 (1.1x)   │
     │         Snap Original vs New with market_ids=None and contention │ 0.361   │ 0.374   │ 0.367   │ 0.323 (1.1x)   │ 0.342 (1.1x)    │ 0.330 (1.1x)   │
     │    Snap Original vs New with specified market_ids and contention │ 0.708   │ 0.722   │ 0.711   │ 0.215 (3.3x)   │ 0.219 (3.3x)    │ 0.217 (3.3x)   │
     └──────────────────────────────────────────────────────────────────┴─────────┴─────────┴─────────┴────────────────┴─────────────────┴────────────────┘
petedmarsh commented 1 year ago

I've run black and formatted the code