search

sourcegraph / zoekt

Fast trigram based code search
Apache License 2.0
598 stars 80 forks source link

Report GCP profiles from zoekt-git-index #816

Closed jtibshirani closed 2 weeks ago

jtibshirani commented 3 weeks ago

This PR initializes the GCP profiler in the zoekt-git-index process so we can examine CPU and memory usage for the indexing process itself.

jtibshirani commented 3 weeks ago

Looking for feedback on this one :) It's a bit weird to be initializing the GCP profiler in a subprocess, especially since GCP collects data every 1 minute for 10 seconds. However, the data looks quite good from local testing:

Screenshot 2024-09-06 at 3 44 43 PM

And in many of the cases we want to improve, indexing can take several minutes, and we can probably learn from this data. So it feels fine to have this as long as we understand the slow sampling rate compared to the process duration. What do you think?