search

wesm / pandas2

Design documents and code for the pandas 2.0 effort.
https://pandas-dev.github.io/pandas2/
306 stars 41 forks source link

More careful management of hash table allocations #35

Open wesm opened 7 years ago

wesm commented 7 years ago

per https://github.com/pydata/pandas/issues/4491

we may consider a fixed-size memory pool (which could be managed with an LRU stack) for hash table data to avoid excess internal index hash tables

chris-b1 commented 7 years ago

xref https://github.com/pydata/pandas/issues/14273 from @ssanderson - this particular case could still be improved in pandas 1.0, but a good example of where hash table size can be problematic.