[FEATURE]: Support Top-level await in `drizzle.config.ts`

[np-kyokyo]

Describe what you want

I want to write a configuration as follows. Here, fetchDatabaseUri is a function that retrieves authentication information from AWS Secrets Manager and returns the database URI.

drizzle.config.tableau.ts

import type { Config } from "drizzle-kit";
import { fetchDatabaseUri } from "./util/rds";

const uri = await fetchDatabaseUri("tableau");

export default {
  schema: "./src/tableau/schema.ts",
  out: "./drizzle/tableau",
  driver: "mysql2",
  dbCredentials: { uri },
} satisfies Config;

Currently, when I run introspect with these settings, I get the following error:

np@MNNP9035-2 ~/g/s/db (db)> bun introspect:dev:tableau
$ dotenv -c dev.ro -- drizzle-kit introspect:mysql --config=drizzle.config.tableau.ts
drizzle-kit: v0.20.14
drizzle-orm: v0.29.4

Custom config path was provided, using 'drizzle.config.tableau.ts'
Reading config file '/Users/np/gitrep/sample/db/drizzle.config.tableau.ts'
node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

Error: Transform failed with 1 error:
/Users/np/gitrep/sample/db/drizzle.config.tableau.ts:4:12: ERROR: Top-level await is currently not supported with the "cjs" output format
    at failureErrorWithLog (/Users/np/gitrep/sample/db/node_modules/esbuild/lib/main.js:1651:15)
    at /Users/np/gitrep/sample/db/node_modules/esbuild/lib/main.js:849:29
    at responseCallbacks.<computed> (/Users/np/gitrep/sample/db/node_modules/esbuild/lib/main.js:704:9)
    at handleIncomingPacket (/Users/np/gitrep/sample/db/node_modules/esbuild/lib/main.js:764:9)
    at Socket.readFromStdout (/Users/np/gitrep/sample/db/node_modules/esbuild/lib/main.js:680:7)
    at Socket.emit (node:events:515:28)
    at addChunk (node:internal/streams/readable:545:12)
    at readableAddChunkPushByteMode (node:internal/streams/readable:495:3)
    at Readable.push (node:internal/streams/readable:375:5)
    at Pipe.onStreamRead (node:internal/stream_base_commons:190:23) {
  errors: [
    {
      detail: undefined,
      id: '',
      location: {
        column: 12,
        file: '/Users/np/gitrep/sample/db/drizzle.config.tableau.ts',
        length: 5,
        line: 4,
        lineText: 'const uri = await fetchDatabaseUri("tableau");',
        namespace: '',
        suggestion: ''
      },
      notes: [],
      pluginName: '',
      text: 'Top-level await is currently not supported with the "cjs" output format'
    }
  ],
  warnings: []
}

Node.js v21.1.0
error: script "introspect:dev:tableau" exited with code 1

[artem-simutin]

I am facing the same issue. I want to fetch db credentials from Azure Key Vault (same as AWS Secret Manager) and cannot do that... 🥲

[bennettcolecohen]

I tried to wrap a promise around it but couldn't get it to work either. Any thoughts?

import type { Config } from "drizzle-kit";
import { getDatabaseConnectionString } from "./src/db/utils.js";

const connectionStringPromise = getDatabaseConnectionString();

const configPromise: Promise<Config> = (async () => {
  const connectionString = await connectionStringPromise;
  return {
    schema: "./src/db/schema.ts",
    out: "./src/db/migrations",
    driver: "pg",
    dbCredentials: {
      connectionString,
    },
  };
})();

export default configPromise;

[kylerush]

I am also having this issue. I am attempting to pull secrets from AWS Secrets manager in my JavaScript and I need top-level await to do that with Drizzle Kit's configuration file. I will have to abandon this strategy and instead use the AWS SDK for Linux to pull and set the environment variables at the OS level. Would be nice for top-level wait to be supported.

[paolostyle]

Workaround:

const getConfig = async () => {
  // simplified
  const dbURL = await getSecret(process.env.DB_CREDENTIALS_SECRET);

  return defineConfig({
    schema: './src/schema.ts',
    out: './drizzle',
    dialect: 'postgresql',
    dbCredentials: {
      url: dbURL,
    },
  });
};

export default getConfig();

But naturally would be great to just use top-level await.

[kylerush]

@paolostyle wouldn't the last line of your code snippet need await?

export default await getConfig();

Since getConfig is an async function, doesn't it need to be awaited?

[paolostyle]

No, it doesn't. You don't ever have to await an async function, even though you probably should in most cases, but this is a config file processed by external code (drizzle-kit in this case). Considering it works, it likely awaits whatever is exported by the config file. I can't check it because a) drizzle-kit isn't open-sourced b) it's quite likely that the config part is handled by an external dependency c) I don't really have time to do that.

That snippet isn't something I would use in my regular code but as I said, it's a workaround.

[alecmev]

Async config works for drizzle-kit migrate/studio but not drizzle-kit generate/check, unfortunately:

ZodError: [
  {
    "code": "invalid_type",
    "expected": "object",
    "received": "promise",
    "path": [],
    "message": "Expected object, received promise"
  }
]

Current (terrible) workaround:

// drizzle.credentials.ts

import { CONNECTION_STRING } from './some-file-with-top-level-await.js';

console.log(CONNECTION_STRING);

// drizzle.config.ts

import { execSync } from 'node:child_process';
import { createRequire } from 'node:module';

import type { Config } from 'drizzle-kit';

const require = createRequire(import.meta.url);

const CONNECTION_STRING = execSync(
  `tsx --no-warnings ${require.resolve('./drizzle.credentials.ts')}`,
  { encoding: 'utf8' },
).trim();

export default {
  dbCredentials: { url: CONNECTION_STRING },
  // ...
} satisfies Config;

[keypuncherlabs]

I'm having the same issue as well, this seems like a very common use case, storing secrets in .env files is hard to share and keep out of version control than just using a better approach like using a secrets manager.

Another workaround which doesn't require a code change is to run the migration while setting the env variable in the same command:

Mac Terminal command:

$ DATABASE_URL='your-db-url' npx drizzle-kit generate

[owlyowl]

Can't wait for this feature... It seems totally essential for any secrets manager.