search

launchdarkly / ld-find-code-refs

Build tool for automatically sending feature flag code references to LaunchDarkly
https://launchdarkly.com
Other
45 stars 34 forks source link

Use Aho-Corasick implementation to speed up searching for extinctions #332

Closed jazanne closed 1 year ago

jazanne commented 1 year ago
  1. This applies Aho-Corasick algorithm used elsewhere to scanning extinctions
  2. It also changes extinction scanning so that it's per file (so it can use the right matcher when monorepo is set up) instead of scanning the entire patch at once.

--

More context -

We have encountered a lot of slow extinctions runs that takes code refs scanning from 1-2 minutes to much longer. One reason for this was a bug we introduced in 2.5.0, but even with the behavior restored, extinction checking can take a really long time and we often advise customer to set a low lookback (default is 10) to make runs faster.

This should dramatically speed up extinction checks and make code refs work better out of the box.

--

so fast 😍

from this...

INFO: 2023/02/16 15:07:36 coderefs.go:182: checking if 1845 flags without references were removed in the last 10
...
INFO: 2023/02/16 15:09:08 coderefs.go:187: found 0 removed flags

to this

INFO: 2023/02/16 14:41:32 coderefs.go:182: checking if 1845 flags without references were removed in the last 10 commits for project: default
...
INFO: 2023/02/16 14:41:33 coderefs.go:187: found 0 removed flags