Parametrize the default database

[francescomucio]

I am using Dataform with BQ. I need to be able to write in a different database (BQ project) depending on the environment.

I was thinking to have something like:

config {
 database = dataform.projectConfig.vars.targetDatabase
 }

But it seems that this is not really possible. Is there another possibility?

[BenBirt]

If you're using Dataform Web, you can use config overrides in your environment: https://docs.dataform.co/dataform-web/scheduling/environments#example-use-separate-databases-for-development-and-production-data

If not, you should definitely still be able to override the database on a per-action basis, see this for an example: https://github.com/dataform-co/dataform/blob/master/examples/common_v2/definitions/override_database_example.sqlx

Can you describe what happens when you try the above?

[francescomucio]

I saw that, but my use case is a bit more complex. I need to write from database A to B, but also A and B depend on the environment. (T

So I have something like:

• dev: write from A to B
• prod: write from C to D

Overriding the database with the config object per-action (which is what I am looking for, because most of the other operations work only in the source databases A or C) doesn't work dynamically. Or at least I didn't find a way to do it.

[BenBirt]

I think I'm a bit confused, can you clarify?

I've just tested the following, and it does work:

whatever.sqlx:

config {
  database: dataform.projectConfig.vars.databaseOverride
}

dataform.json (similar for environments.json):

{
    ...
    "vars": {
        "databaseOverride": "hi"
    }
}

[francescomucio]

Let me test it, maybe I was doing it wrong...

[francescomucio]

You were right. I think that was the first thing we tried but we got the variable wrong. So we assumed that was not working :)