search

googleapis / google-cloud-cpp

C++ Client Libraries for Google Cloud Services
https://cloud.google.com/
Apache License 2.0
554 stars 373 forks source link

API to unpack Row dynamically #7225

Closed zuguangz closed 1 year ago

zuguangz commented 3 years ago

What component of google-cloud-cpp is this feature request for? Spanner

Is your feature request related to a problem? Please describe. I construct the SQL sent to Spanner, according to various options from the caller, affecting the actual columns (both names and types) getting returned from the SQL. The C++ API requires me to define a struct or tuple in order to unpack data from Row (as returned by Client::ExecuteQuery and other APIs), but it is infeasible for me to write rigid C++ struct or tuple types for all the possible combinations of the various options callers may pass in

The Row API does support retrieving data of Value type, but Value itself does not have any API to work with it dynamically, especially when the Value holds a STRUCT. To unpack a STRUCT typed value, it needs a C++ tuple again

Describe the solution you'd like A set of reflection based APIs would help solve the need. For example, Value may return a vector of "StructField" objects for STRUCT type, and each StructField object holds a Value, and a type if Value itself does not contain type information (enum or a more dynamic data structure for ARRAYs and STRUCTs)

One concrete illustration would be zetasql's Value and Type: https://github.com/google/zetasql/blob/master/zetasql/public/value.h and https://github.com/google/zetasql/blob/master/zetasql/public/type.h

Describe alternatives you've considered Write code to generate code to define C++ struct and tuple for every combination of the client options. Not scalable because the number of concrete classes to generate goes exponential with the number of options (as it needs to cover arbitrary combinations), and likely make compilation slow or downright impossible

The other alternative is to call internal::FromProto and internal::ToProto (https://github.com/googleapis/google-cloud-cpp-spanner/blob/master/google/cloud/spanner/value.h), figure out implementation details of the actual encoding of various types, and risk any backward incompatible change made there to break my application

Additional context Add any other context about the feature request here.

devjgm commented 2 years ago

Still thinking. Keeping open.

coryan commented 2 years ago

Strike 2. @devbww to follow up.

devbww commented 1 year ago

For the record, internal discussions are in http://g/cloud-cxx-team/MovmxIUAK-I/N6M1krQ8AgAJ.

For the time being, Zuguang is continuing to use internal APIs like google::cloud::spanner_internal::ToProto() (see http://cl/516374011), which we have committed to supporting until there is a better solution.

coryan commented 1 year ago

Sounds like we should close this. Please reopen if you disagree.