search

apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.81k stars 4.23k forks source link

Incorrect Spanner IO Request Count metrics #21635

Open damccorm opened 2 years ago

damccorm commented 2 years ago

IO request count metrics calculated incorrectly for GCP Spanner

 

Resource ID is formulated incorrectly

Spanner Table:

//spanner.googleapis.com/projects/\{projectId}/{*}topics{*}/\{databaseId}/tables/\{tableId}

should be

//spanner.googleapis.com/projects/\{projectId}/instances/\{instanceId}/databases/\{databaseId}/tables/\{tableId}

and is populated incorrectly – instance ID is used in place of tableID

Spanner SQL Query:

//spanner.googleapis.com/projects/\{projectId}/queries/\{queryName}  should be {}//spanner.googleapis.com/projects/\{projectId}/{}{}instances/\{instanceId}/queries{}{}/\{queryName} {}

and queryName is nullable which cause issued downstream this is not actually populated at all - queries are logged as reads on an instance. 

Imported from Jira BEAM-14121. Original Jira may contain additional context. Reported by: nielm.

damccorm commented 2 years ago

Unable to assign user @nielm. If able, self-assign, otherwise tag @damccorm so that he can assign you. Because of GitHub's spam prevention system, your activity is required to enable assignment in this repo.