Closed dreo closed 6 months ago
Thats not possible :D
@vutran1710 Yes, there really is a gigantic performance drop introduced with 3.0. But with the latest 3.3 that became from unbearable to just 30% compared with 2.0. It seems your latest PR !158 has fixed it. It is still not up to 2.0 levels, but it is probably acceptable.
@vutran1710 Yes, there really is a gigantic performance drop introduced with 3.0. But with the latest 3.3 that became from unbearable to just 30% compared with 2.0. It seems your latest PR !158 has fixed it. It is still not up to 2.0 levels, but it is probably acceptable.
Thats so strange. Can you share with me some snippet so i can try it myself?
For 2:
import timeit
from pyrate_limiter import Duration, Limiter, RequestRate
hourly_rate = RequestRate(500000000, Duration.HOUR)
daily_rate = RequestRate(1000000000, Duration.DAY)
monthly_rate = RequestRate(10000000000, Duration.MONTH)
limiter = Limiter(hourly_rate, daily_rate, monthly_rate)
numbers = [100, 1_000, 10_000, 100_000, 1_000_000]
for XXX in numbers:
timed = timeit.timeit(stmt=lambda: limiter.try_acquire("id"), number=XXX)
print(f'limiter.try_acquire {XXX}: {timed}')
For 3:
import timeit
from pyrate_limiter import Duration, InMemoryBucket, Limiter, Rate
hourly_rate = Rate(500000000, Duration.HOUR)
daily_rate = Rate(1000000000, Duration.DAY)
monthly_rate = Rate(10000000000, Duration.WEEK)
limiter = Limiter(InMemoryBucket([hourly_rate, daily_rate, monthly_rate]))
numbers = [100, 1_000, 10_000, 100_000, 1_000_000]
for XXX in numbers:
timed = timeit.timeit(stmt=lambda: limiter.try_acquire("id"), number=XXX)
print(f'limiter.try_acquire {XXX}: {timed}')
Results:
2.10.0:
limiter.try_acquire 100: 0.0008874609999338645
limiter.try_acquire 1000: 0.008765025000002424
limiter.try_acquire 10000: 0.09226327299995774
limiter.try_acquire 100000: 0.8978742110000439
limiter.try_acquire 1000000: 8.994104164999953
3.2.1
limiter.try_acquire 100: 0.0023891220000678004
limiter.try_acquire 1000: 0.03898568099998556
limiter.try_acquire 10000: 1.3275317239999822
limiter.try_acquire 100000: 285.53246885500005
^C
3.3.0
limiter.try_acquire 100: 0.0012090879999959725
limiter.try_acquire 1000: 0.012968415000045752
limiter.try_acquire 10000: 0.12044174899995141
limiter.try_acquire 100000: 1.212711079000087
limiter.try_acquire 1000000: 12.330737776000092
(Debian 12, Python 3.12.1)
Please upgrade to the latest version (v3.4)
I think you will be pleased.
And thanks a lot for helping out with the snippets and all.
After migrating to 3.x, I've noticed that PyrateLimiter does not scale lineary with request counts anymore (unlike 2.x).
Consider this snipper:
Results for 3x:
number=1000: 0.024325600010342896 number=10000: 0.8553861999535002 number=100000: 75.72125629999209 number=1000000: does not finish in 10 minutes
Results for 2x:
number=1000000: 0.1329616999719292