dataform-co / dataform

Dataform is a framework for managing SQL based data operations in BigQuery
https://cloud.google.com/dataform/docs
Apache License 2.0
810 stars 151 forks source link

Assertions are not executed properly #1756

Open kajen3 opened 1 month ago

kajen3 commented 1 month ago

I've created a simple .sqlx file with the following content inside the definitions folder:

config {
    type: "table",
    schema: "dev",
    name: "revenue",
    assertions: {
        uniqueKeys: [
            ["date"],
            ["store"]
        ]
    }
}

SELECT
  date,
  store,
  net_revenue,
  net_revenue_food
FROM
  ${ref("store_revenue_view")}

While the query itself works fine, it seems that the assertions are not executed properly. The error message says:

Invalid value: Invalid dataset ID "...b0a.". Dataset IDs must be alphanumeric (plus underscores) and must be at most 1024 characters long. at [2:3] at [8:5] 

The dataset id is alphanumeric, but I've spotted a difference in the second line of the executed code. The code that fails (assertions) looks as follows:

BEGIN
  CREATE SCHEMA IF NOT EXISTS `...b0a.` OPTIONS(location="europe-west1");
...

while the first two lines of the job itself look as follows:

BEGIN
  CREATE SCHEMA IF NOT EXISTS `...b0a.dev` OPTIONS(location="europe-west1");
...