search

carbon-language / carbon-lang

Carbon Language's main repository: documents, design, implementation, and related tools. (NOTE: Carbon Language is experimental; see README)
http://docs.carbon-lang.dev/
Other
32.25k stars 1.48k forks source link

Reserve memory for the identifiers hashtable. #4107

Closed chandlerc closed 2 months ago

chandlerc commented 3 months ago

This uses a heuristic reserve to greatly reduce hashtable growth of the identifiers hashtable. The design of the hashtable itself is optimized around compact memory use and is especially slow to grow and so this has an outsized impact.

The heuristic was computed using scripts/source_stats.py and looking at C++ codebases. We may want to periodically re-evaluate it as Carbon code emerges and we have better data on its distributions of tokens.

This also required fixing the Reserve method on CanonicalValueStore that wasn't actually used anywhere and so didn't even compile correctly. I added it to the relevant unit test so it is at least compiled locally to its definition.

chandlerc commented 2 months ago

Ping? Just looking for confirmation that moving the comment out-of-line to a separate area and cleaning up things was enough that this LG for landing.

jonmeow commented 2 months ago

I believe we'd discussed this in person a couple weeks ago, I'm fine with you merging.