search

Smattr / clink

a modern re-implementation of Cscope
The Unlicense
43 stars 2 forks source link

high contention #112

Closed Smattr closed 1 year ago

Smattr commented 2 years ago

Sampling on an 8-way machine while scanning a medium sized code base, indicates a lot of inter-thread contention. I haven't looked into the details yet, but I suspect it's database insertion. When 8+ threads are parsing code, they're calling clink_db_add_symbol quite a lot.

The obvious solution to this would be to buffer insertions in memory until the end of parsing the current file, then flush them to the database in a batch. We could even wrap this in a lock+transaction for presumably an additional performance improvement.

Smattr commented 1 year ago

All of the ideas floated above were implemented in 12e8e3113855a98cf9dd32bd2c6ca4688ca2f4fa, 15d4bcc6b80b832c3b5192b7b6f015337e3705c2, and 7196725910c2bba3c1fe69a9bad58993b6a36dc5, respectively. They did indeed accelerate things.