search

googleapis / google-cloud-cpp-spanner

C++ client library for Google Cloud Spanner
https://cloud.google.com/spanner/
Apache License 2.0
29 stars 14 forks source link

Implement proto Value "unchunker" #161

Closed devjgm closed 5 years ago

devjgm commented 5 years ago

According to Spanner's ResultSet proto, the PartialResultSet message that is returned from streaming operations may have "chunked" values. These are conceptually single values that may actually be split across several adjacent "value" proto bufs. See the docs here about how the chunking works.

Well, since results may be chunked, we need an "unchunker". That is, we need something that can take a "stream" of proto Values and recombine them into the single logical Value that they represent.

devjgm commented 5 years ago

Per an offline chat the interface that @mr-salty would like is something along the lines of

bool MergeChunks(Value& existing, Value const& chunk);
mr-salty commented 5 years ago

I was just offering that as one possibility, but essentially that's the situation - we'll have an existing partial Value, and a new Value that came in off the wire to be merged into it. IDK if it's also necessary for the chunker to know whether the new Value is the last chunk or if there are more to come.

devjgm commented 5 years ago

I think it's a good suggestion. I don't think the unchunker needs to know if the new chunk is the last one. It will just merge any two values you give it. If it can't (due to wrong types, etc.) it'll return false.

Note: I'll probably use this slightly modified signature:

bool MergeChunk(Value& value, Value&& chunk);

So that we can "consume" the chunk as it gets merged into the value argument.