lsif-clang: Reduce contention by using per-table mutexes.

sourcegraph / lsif-clang

Language Server Indexing Format (LSIF) generator for C, C++ and Objective C

https://lsif.dev/

34 stars 8 forks source link

lsif-clang: Reduce contention by using per-table mutexes. #85

Closed varungandhi-src closed 2 years ago

varungandhi-src commented 2 years ago

For an indexing job of CMake on a 60 core machine:

Before this change: indexing time 30s with about 31/60 core utilization
After this change: indexing time 25s with about 42/60 core utilization

There doesn't seem to be much effect on 16 core machines.

This was changed in upstream LLVM too (in clangd). The code change to IndexerMain.cpp is redundant, because the code is duplicated in LSIFClangMain.cpp, but we keep the two in sync.

Test plan

Manually tested

varungandhi-src commented 2 years ago

Looks like there was another optimization around files in the upstream commit which introduced this.

https://github.com/llvm/llvm-project/commit/dad804a193edf092322d80bb404fabb2f6f2c888

varungandhi-src commented 2 years ago

Updated benchmarks with the file optimization: (different 60 core instance)

Without any optimization: 34s, 403M index
With separate mutexes: 29s, 403M index
With file uniquing+skipping: 17s, 390M index