Remove line matches from Zoekt

[camdencheek]

Once we're reasonably comfortable with the using chunk matches instead of line matches in Zoekt, we should remove the now-unused line match code. This is not a small task since all the tests and stuff also depend on that representation.

/cc @sourcegraph/search-platform

[keegancsmith]

I'm not totally sure we want to do this, since I believe other consumers of the zoekt repo use linematches. A more conservative thing we could do is make chunkmatches the default and maybe just mark the line matches field as deprecated so tools start to complain?

Either way, not so long ago I removed the duplicate ranking paths we had for chunkmatches and linematches which too me was the largest part of the debt of both paths.

[camdencheek]

If we can clean up the duplicate codepaths and just leave this at the API layer, sounds reasonable to me to just keep it around. Besides, line matches are still easier to use for simple situations.

[keegancsmith]

Right so I took another look. Here are the places we do different things depending on ChunkMatches:

• How we merge candidates :: ChunkMatches drops while LineMatches merges. I propose we use the same behaviour. I don't know what is best, there are nice things about both approaches. I suspect ChunkMatches is easier to reason about, but maybe visually line matches is more aligned with user expectations (eg think grep output)
• The fill logic :: this is the code that will always exist. IE how we take internal candidateMatch representation into the API representation.
• NewDisplayTruncator in shards pkg :: this is operating at the API representation since it is behaving at a higher level than per shard search. This is an example of extra complexity that we could remove. However, the code all lives in one file, is well tested and is decoupled. So not that big of win from a mental model perspective.
• GRPC layer supports both.

And thats it... I reckon we could change merge candidate behaviour and drop support for line matches in grpc. Keep LineMatch default, but grpc layer enforces chunkmatches for sourcegraph. I think that would be ideal from the perspective of keeping a simpler representation (for tests as well), and isn't that heavy of a cost.

There is the option of exercising LineMatches from the go API totally, and introducing a convertor in zoekt/web pkg for REST. However, TBH that is quite a big change given so many tests depend on linematches right now.