insights: batch gitserver network calls

sourcegraph / sourcegraph-public-snapshot

Code AI platform with Code Search & Cody

https://sourcegraph.com

Other

10.1k stars 1.28k forks source link

insights: batch gitserver network calls #33864

Open coury-clark opened 2 years ago

coury-clark commented 2 years ago

Notebook with some context: https://sourcegraph.com/notebooks/Tm90ZWJvb2s6NjUy

Reduce the number of gitserver interactions from the Insights background process by batching the requests per repository.

sourcegraph-bot-2 commented 2 years ago

Heads up @joelkw @felixfbecker @vovakulikov @unclejustin - the "team/code-insights" label was applied to this issue.

github-actions[bot] commented 2 years ago

Hey, @sourcegraph/batchers (@eseliger @LawnGnome @courier-new @adeola-ak @BolajiOlajide @malomarrec @chrispine @danielmarquespt) - we have been mentioned. Let's take a look.

chwarwick commented 2 years ago

I looked at the two calls a since the firstEverCommit is already cached I was thinking wrapping the findRecentCommit to first attempt to pull it from the commit_index in postgres and then fall back to calling the gitserver if the index isn't sufficiently up to date could provide a greater decrease in the calls to gitserver.