search

viralogic / py-enumerable

A Python module used for interacting with collections of objects using LINQ syntax
MIT License
187 stars 24 forks source link

Performance Issue on Select #38

Closed viralogic closed 4 years ago

viralogic commented 4 years ago

I received an email stating that a simple select query using py-linq on 80,000 entries is very slow when compared to versions pre-1.2.0.

The cause of this is not immediately apparent to me, but warrants further investigation as 1.2.0 was the release that started using underlying cycle itertool method so that Enumerable objects could be iterated over more than once.

viralogic commented 4 years ago

Fixed in version 1.2.2. The actual issue was the call to_list which was taking entirely too long. Wrote some regression tests that monitor the performance of this call (and a few others) to ensure that they are within 5x of their underlying python iterator calls as a minimum baseline.