Add Flush and BufferGet requests for pipelined DML

[ekexium]

Ref https://github.com/tikv/tikv/issues/16291 Don't merge it until corresponding work finishes.

[you06]

Need to rebase master, I cannot build the latest TiDB with this PR because some protocols are missing.

[cfzjywxk]

Another way for reading a transaction's own uncommitted data in tikv:

• kv-client: For pipelined-dml read, adding a flag in the existing BatchGet request, it indicates whether the current read is membuf read in tikv or union store read in tikv.

• tikv: reuse the same BatchGet interface.

  • storage: implement the buffer_batch_get and union_store_batch_get methods.

So we could avoid introducing the BufferGet interface and do union store get within one RPC request.

[ekexium]

Another way for reading a transaction's own uncommitted data in tikv:

• kv-client: For pipelined-dml read, adding a flag in the existing BatchGet request, it indicates whether the current read is membuf read in tikv or union store read in tikv.

• tikv: reuse the same BatchGet interface.

  • storage: implement the buffer_batch_get and union_store_batch_get methods.

So we could avoid introducing the BufferGet interface and do union store get within one RPC request.

So the only difference is in the grpc level, whether to use 3 different RPCs for snapshot read, buffer read, and union read; or use one RPC for all of them and use flags to differentiate.

Seems a trade-off between interface simplicity and clarity. Imagine when we see a batch_get from logs or metrics, we would have no idea what it is reading.

[cfzjywxk]

We essentially need to support one RPC to complete the membuf read + snapshot read, regardless of whether or not a separate KV interface is split out. This scalability needs to be reserved. It doesn't seem to introduce too much protocol complexity to separately split out BufferBatchGetRequest, although it introduces a new interface. However, I don't think we need to introduce another UnionStoreBatchGetRequest interface.

[cfzjywxk]

@MyonKeminta PTAL

[MyonKeminta]

It doesn't seem to introduce too much protocol complexity to separately split out BufferBatchGetRequest, although it introduces a new interface. However, I don't think we need to introduce another UnionStoreBatchGetRequest interface.

I agree with this. We can regard it as an RPC interface that supports reading with the buffer, including buffer reading only and mixed (union) reading.

But my problem is whether the word "buffer" is proper here...

[ekexium]

@MyonKeminta any suggestions on names?