mozilla / bigquery-etl

Bigquery ETL
https://mozilla.github.io/bigquery-etl
Mozilla Public License 2.0
253 stars 100 forks source link

bug 1904841: fix sql udf cast_json breaking w string keys #5845

Closed edugfilho closed 3 months ago

edugfilho commented 3 months ago

fixes 1904841 Checklist for reviewer:

For modifications to schemas in restricted namespaces (see CODEOWNERS):

┆Issue is synchronized with this Jira Task

dataops-ci-bot commented 3 months ago

Integration report for "fix sql udf cast_json breaking w string keys"

sql.diff

Click to expand! ```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/mozfun/glam/histogram_cast_json/udf.sql /tmp/workspace/generated-sql/sql/mozfun/glam/histogram_cast_json/udf.sql --- /tmp/workspace/main-generated-sql/sql/mozfun/glam/histogram_cast_json/udf.sql 2024-06-26 13:34:10.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/mozfun/glam/histogram_cast_json/udf.sql 2024-06-26 13:34:05.000000000 +0000 @@ -16,7 +16,12 @@ COALESCE( CONCAT( '{', - STRING_AGG(CONCAT('"', key, '":', ROUND(value, 4)) ORDER BY CAST(key AS FLOAT64)), + STRING_AGG( + CONCAT('"', key, '":', ROUND(value, 4)) + ORDER BY + COALESCE(SAFE_CAST(key AS FLOAT64)), + key + ), '}' ), "{}" @@ -45,3 +50,9 @@ ) ), assert.equals('{}', glam.histogram_cast_json(ARRAY>[])), + assert.equals( + '{"always":0.5,"never":0.5}', + glam.histogram_cast_json( + ARRAY>[("always", 0.5), ("never", 0.5)] + ) + ), ```

Link to full diff

dataops-ci-bot commented 3 months ago

Integration report for "Merge branch 'main' into fix-udf-cast-json-string"

sql.diff

Click to expand! ```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/mozfun/glam/histogram_cast_json/udf.sql /tmp/workspace/generated-sql/sql/mozfun/glam/histogram_cast_json/udf.sql --- /tmp/workspace/main-generated-sql/sql/mozfun/glam/histogram_cast_json/udf.sql 2024-06-26 17:28:42.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/mozfun/glam/histogram_cast_json/udf.sql 2024-06-26 17:28:53.000000000 +0000 @@ -16,7 +16,12 @@ COALESCE( CONCAT( '{', - STRING_AGG(CONCAT('"', key, '":', ROUND(value, 4)) ORDER BY CAST(key AS FLOAT64)), + STRING_AGG( + CONCAT('"', key, '":', ROUND(value, 4)) + ORDER BY + COALESCE(SAFE_CAST(key AS FLOAT64)), + key + ), '}' ), "{}" @@ -45,3 +50,9 @@ ) ), assert.equals('{}', glam.histogram_cast_json(ARRAY>[])), + assert.equals( + '{"always":0.5,"never":0.5}', + glam.histogram_cast_json( + ARRAY>[("always", 0.5), ("never", 0.5)] + ) + ), ```

Link to full diff