Unify search latency, duration, and event collection in `LogJob`

[camdencheek]

The stream and GraphQL APIs used to work very differently, so they have very different metrics and event logging. However, now the streaming API and GraphQL API both use the same underlying machinery, so it makes sense to unify the metrics and event logging so that the numbers are directly comparable.

What would this look like?

• Move logLatency into LogJob.Run()
• Merge logSearch and logBatch, moving them into LogJob.
• You will probably want to use NewResultCountingStream and NewStatsObservingStream to get the required info for logging
• You'll probably need to merge some prometheus metrics, which means updating our Grafana dashboards

This also has the effect of having good observability on all searches, even ones from weird sources like the Stats endpoint or code monitors.

[github-actions[bot]]

Hey, @sourcegraph/batchers (@eseliger @LawnGnome @courier-new @adeola-ak @BolajiOlajide @Piszmog @malomarrec @chrispine @danielmarquespt) - we have been mentioned. Let's take a look.

[camdencheek]

lol, sorry batchers, just a function name

[eseliger]

You’ll have to take our opinions now 😇

[camdencheek]

I'll always take the opinion of an /Eri(c|k)/

[jtibshirani]

I'm going to unassign myself, since I'm thoroughly onboarded by now and someone else can benefit from this well-scoped idea :)