dataform-co / dataform

Dataform is a framework for managing SQL based data operations in BigQuery
https://cloud.google.com/dataform/docs
Apache License 2.0
811 stars 152 forks source link

Running the CLI with "run --dry-run" causes BigQuery changes to be applied #1739

Closed kolban-google closed 1 month ago

kolban-google commented 1 month ago

Summary: When I run dataform run --dry-run, the SQL is executed. I had anticipated the SQL not to be executed.

Details: I have a BigQuery dataset I call dataform. It has no tables in it.

I have a SQLX file called ages.sqlx. Contained within is:

config { type: "table" }

SELECT
    name, age
FROM 
(
    SELECT "Bob" as name, 55 as age UNION ALL
    SELECT "Jane", 42 UNION ALL
    SELECT "John", 35 UNION ALL
    SELECT "Sarah", 27 UNION ALL
    SELECT "Mike", 57
) 

I then run:

$ dataform run --dry-run
Compiling...

Compiled successfully.

Dry running (no changes to the warehouse will be applied)...
Table dry run success:  dataform.ages [table]
$ 

I then query my BigQuery environment and now find that a table called dataform.ages was indeed created. I had not expected the table to be created during the dry run invocation.

My CLI version is:

$ dataform --version
3.0.0-beta.5
$