dataform-co / dataform

Dataform is a framework for managing SQL based data operations in BigQuery
https://cloud.google.com/dataform/docs
Apache License 2.0
848 stars 160 forks source link

Nested protobufs aren't created during protobuf creation #607

Closed Ekrekr closed 4 years ago

Ekrekr commented 4 years ago

Have looked for solutions but haven't found anything nice so putting as an issue. Could be this belongs as an issue in protobufjs.

Example, where bigquery is of type BigQueryOptions, which has members public partitionBy: string; and public clusterBy: string[];:

const tmp = dataform.Table.create({
    bigquery: {
        partitionBy: "DATE(test)"
    }
});

console.log(tmp)

outputs

Table {
    ...
    bigquery: { partitionBy: 'DATE(test)' }
}

whereas it what is actually received by the RPC is

Table {
    ...
    bigquery: { partitionBy: 'DATE(test)', clusterBy: [] }
}

Which can cause unpredictable behaviour, and recently did cause an issue within the API tests.

lewish commented 4 years ago

Thanks Elias, sorry going to have to close this bug as I don't think there is anything we can feasibly do about this. This is kind of one of those things when working with protos unfortunately!