search

dataform-co / dataform

Dataform is a framework for managing SQL based data operations in BigQuery
https://cloud.google.com/dataform/docs
Apache License 2.0
834 stars 161 forks source link

When using ref() referencing an action which isn't a view/table or hasOutput, the error message is not helpful #342

Closed BenBirt closed 5 years ago

BenBirt commented 5 years ago

To recreate (on redshift at least)

Create a file dataset_1.sqlx and leave it empty

Create a file dataset_2.sqlx with " SELECT * FROM ${ref("dataset_1")}.

You get compile errors similar to the following (~25 lines): Cannot read property 'schema' of null TypeError: Cannot read property 'schema' of null at RedshiftAdapter.resolveTarget (node_modules/@dataform/core/adapters/redshift.js:21:31)

On the other hand, if dataset_1.sqlx doesn't exist, the UI shows a beautiful validation error: Missing dependency detected: Node "dataset_2" depends on "dataset_1" which does not exist.

I think both errors should lead to the same error messages

Moved from dataform-co/dataform-co#1099 /cc @G2H

BenBirt commented 5 years ago

Copying from https://github.com/dataform-co/dataform-co/issues/1099

""" Note for future issue fixers: This is likely because we changed operations to no longer have 'target' set unless they specify 'hasOutput'. """

BenBirt commented 5 years ago

Should be fixed in https://github.com/dataform-co/dataform/pull/352