greatexpectationslabs / ge_tutorials

Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.
167 stars 84 forks source link

Unable to find match for config variable warehouse #1

Open KristinaPlazonic opened 4 years ago

KristinaPlazonic commented 4 years ago

Hi,

trying out the demo, and am running into a problem when executing the following (btw, airflow test command required me to add execution date to dag and task):

airflow test ge_tutorials_dag task_validate_source_data 2020-03-09

This command gives me the following error

  File "/home/kris/anaconda3/lib/python3.7/site-packages/great_expectations/data_context/util.py", line 168, in substitute_config_variable
    raise InvalidConfigError("Unable to find match for config variable {:s}. See https://great-expectations.readthedocs.io/en/latest/reference/data_context_reference.html#managing-environment-and-secrets".format(match.group(1)))
great_expectations.exceptions.InvalidConfigError: Unable to find match for config variable warehouse. See https://great-expectations.readthedocs.io/en/latest/reference/data_context_reference.html#managing-environment-and-secrets

I'm doing mariadb from fresh install on my local machine (user root and no pwd) and my .dbt profile is:

$ cat ~/.dbt/profiles.yml 

# For more information on how to configure this file, please see:
# https://docs.getdbt.com/docs/profile

default:
  outputs:
    dev:
      type: mysql
      threads: 1
      host: 127.0.0.1
      port: 3306
      user: root
      pass: 
      dbname: warehouse

and I do have env vars set up:

GE_TUTORIAL_DB_URL=mysql://root@localhost:3306/ge    #also tried warehouse as db name instead of ge
GE_TUTORIAL_PROJECT_PATH=/home/kris/projects/ge_demo/ge_tutorials

(and airflow is setup correctly because I could see ge_tutorials_dag in the airflow ui)

Sorry, don't have time to investigate further, just wanted to give feedback to your awesome tutorial, maybe it helps also someone else if they hit a snag.

Thanks and keep up your awesome stuff :D