L-Mario564 / drizzle-dbml-generator

Generate DBML markup from your schema defined with Drizzle ORM.
MIT License
160 stars 8 forks source link

Extra Config Builder chokes if there are not extra config columns. #14

Closed johnkraczek closed 2 months ago

johnkraczek commented 4 months ago

I found this library and thought it was very cool, so thanks for making it. 👍 😁

I ran into an issue and wanted to report it, as well as my current workaround. It seems to be an issue with the extraConfigBuilder & extraConfigColumns

Let me get into my setup.

My schema has next auth tables directly from what they recommend in their documentation for example the Account Provider Table that looks like this:

schemas/provider-account.ts (drizzle schema table) ``` import { integer, primaryKey, text } from "drizzle-orm/pg-core"; import { createTable, defaultFields } from "../../utils"; import type { AdapterAccount } from "@auth/core/adapters"; import { users } from "~/server/db/schemas/users"; import { relations } from "drizzle-orm"; // =================== TABLE DEFINITION =================== export const accounts = createTable( "account", { userId: text("userId") .notNull() .references(() => users.id, { onDelete: "cascade" }), type: text("type").$type().notNull(), provider: text("provider").notNull(), providerAccountId: text("providerAccountId").notNull(), refresh_token: text("refresh_token"), access_token: text("access_token"), expires_at: integer("expires_at"), token_type: text("token_type"), scope: text("scope"), id_token: text("id_token"), session_state: text("session_state"), email: text("email"), accountName: text("accountName"), accountImage: text("accountImg"), ...defaultFields(), }, (account) => ({ compoundKey: primaryKey({ columns: [account.provider, account.providerAccountId], }), }), ); // =================== RELATIONSHIPS =================== export const accountsRelations = relations(accounts, ({ one }) => ({ user: one(users, { fields: [accounts.userId], references: [users.id] }), })); ```
utils/dbml.ts (the file to define my drizzle setup and generate the DBML) ``` import * as schema from "./schema"; import { pgGenerate } from "drizzle-dbml-generator"; // Using Postgres for this example console.debug("🟢 Generating DBML"); console.log(Object.keys(schema)); try { pgGenerate({ schema, out: "./schema.dbml", relational: true, }); console.info("🟢 DBML generated"); } catch (e) { console.error("🔴 Error generating DBML"); console.error("Error generating DBML:", e); process.exit(1); } ```

When I run the code with that table included in the schema then it chokes with this error:

$ node --import tsx --env-file .env src/server/db/utils/dbml/index.ts
🟢  Generating DBML
Generating DBML...
[ 'accounts', 'accountsRelations' ]
🔴  Error generating DBML
Error generating DBML: TypeError: Cannot read properties of undefined (reading 'name')
    at /Users/john/Development/drizzle-dbml-generator/dist/index.cjs:76:36
    at Array.map (<anonymous>)
    at wrapColumns (/Users/john/Development/drizzle-dbml-generator/dist/index.cjs:76:13)
    at PgGenerator.generateTable (/Users/john/Development/drizzle-dbml-generator/dist/index.cjs:205:29)
    at PgGenerator.generate (/Users/john/Development/drizzle-dbml-generator/dist/index.cjs:289:35)
    at pgGenerate (/Users/john/Development/drizzle-dbml-generator/dist/index.cjs:333:68)
    at <anonymous> (/Users/john/Development/ydtb/src/server/db/utils/dbml/index.ts:8:3)
    at ModuleJob.run (node:internal/modules/esm/module_job:222:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:316:24)
    at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:123:5)
error: script "db:dbml" exited with code 1
Console log of the table object (common.ts around line 134) ``` table: PgTable { userId: PgText { name: 'userId', primary: false, notNull: true, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_userId_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'userId', notNull: true, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_userId_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, type: PgText { name: 'type', primary: false, notNull: true, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_type_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'type', notNull: true, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_type_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, provider: PgText { name: 'provider', primary: false, notNull: true, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_provider_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'provider', notNull: true, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_provider_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, providerAccountId: PgText { name: 'providerAccountId', primary: false, notNull: true, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_providerAccountId_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'providerAccountId', notNull: true, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_providerAccountId_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, refresh_token: PgText { name: 'refresh_token', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_refresh_token_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'refresh_token', notNull: false, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_refresh_token_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, access_token: PgText { name: 'access_token', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_access_token_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'access_token', notNull: false, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_access_token_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, expires_at: PgInteger { name: 'expires_at', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_expires_at_unique', uniqueType: undefined, dataType: 'number', columnType: 'PgInteger', enumValues: undefined, config: { name: 'expires_at', notNull: false, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_expires_at_unique', uniqueType: undefined, dataType: 'number', columnType: 'PgInteger' }, table: [Circular *1] }, token_type: PgText { name: 'token_type', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_token_type_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'token_type', notNull: false, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_token_type_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, scope: PgText { name: 'scope', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_scope_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'scope', notNull: false, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_scope_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, id_token: PgText { name: 'id_token', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_id_token_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'id_token', notNull: false, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_id_token_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, session_state: PgText { name: 'session_state', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_session_state_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'session_state', notNull: false, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_session_state_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, email: PgText { name: 'email', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_email_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'email', notNull: false, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_email_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, accountName: PgText { name: 'accountName', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_accountName_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'accountName', notNull: false, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_accountName_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, accountImage: PgText { name: 'accountImg', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_accountImg_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: { name: 'accountImg', notNull: false, default: undefined, hasDefault: false, primaryKey: false, isUnique: false, uniqueName: 'dv_account_accountImg_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined }, table: [Circular *1] }, createdAt: PgTimestamp { name: 'createdAt', primary: false, notNull: false, default: SQL { decoder: [Object], shouldInlineParams: false, queryChunks: [Array] }, defaultFn: undefined, hasDefault: true, isUnique: false, uniqueName: 'dv_account_createdAt_unique', uniqueType: undefined, dataType: 'date', columnType: 'PgTimestamp', enumValues: undefined, config: { name: 'createdAt', notNull: false, default: [SQL], hasDefault: true, primaryKey: false, isUnique: false, uniqueName: 'dv_account_createdAt_unique', uniqueType: undefined, dataType: 'date', columnType: 'PgTimestamp', withTimezone: false, precision: undefined }, table: [Circular *1], withTimezone: false, precision: undefined, mapFromDriverValue: [Function: mapFromDriverValue], mapToDriverValue: [Function: mapToDriverValue] }, updatedAt: PgTimestamp { name: 'updatedAt', primary: false, notNull: false, default: SQL { decoder: [Object], shouldInlineParams: false, queryChunks: [Array] }, defaultFn: undefined, hasDefault: true, isUnique: false, uniqueName: 'dv_account_updatedAt_unique', uniqueType: undefined, dataType: 'date', columnType: 'PgTimestamp', enumValues: undefined, config: { name: 'updatedAt', notNull: false, default: [SQL], hasDefault: true, primaryKey: false, isUnique: false, uniqueName: 'dv_account_updatedAt_unique', uniqueType: undefined, dataType: 'date', columnType: 'PgTimestamp', withTimezone: false, precision: undefined }, table: [Circular *1], withTimezone: false, precision: undefined, mapFromDriverValue: [Function: mapFromDriverValue], mapToDriverValue: [Function: mapToDriverValue] }, [Symbol(drizzle:Name)]: 'dv_account', [Symbol(drizzle:OriginalName)]: 'dv_account', [Symbol(drizzle:Schema)]: undefined, [Symbol(drizzle:Columns)]: { userId: PgText { name: 'userId', primary: false, notNull: true, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_userId_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, type: PgText { name: 'type', primary: false, notNull: true, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_type_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, provider: PgText { name: 'provider', primary: false, notNull: true, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_provider_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, providerAccountId: PgText { name: 'providerAccountId', primary: false, notNull: true, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_providerAccountId_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, refresh_token: PgText { name: 'refresh_token', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_refresh_token_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, access_token: PgText { name: 'access_token', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_access_token_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, expires_at: PgInteger { name: 'expires_at', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_expires_at_unique', uniqueType: undefined, dataType: 'number', columnType: 'PgInteger', enumValues: undefined, config: [Object], table: [Circular *1] }, token_type: PgText { name: 'token_type', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_token_type_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, scope: PgText { name: 'scope', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_scope_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, id_token: PgText { name: 'id_token', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_id_token_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, session_state: PgText { name: 'session_state', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_session_state_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, email: PgText { name: 'email', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_email_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, accountName: PgText { name: 'accountName', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_accountName_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, accountImage: PgText { name: 'accountImg', primary: false, notNull: false, default: undefined, defaultFn: undefined, hasDefault: false, isUnique: false, uniqueName: 'dv_account_accountImg_unique', uniqueType: undefined, dataType: 'string', columnType: 'PgText', enumValues: undefined, config: [Object], table: [Circular *1] }, createdAt: PgTimestamp { name: 'createdAt', primary: false, notNull: false, default: [SQL], defaultFn: undefined, hasDefault: true, isUnique: false, uniqueName: 'dv_account_createdAt_unique', uniqueType: undefined, dataType: 'date', columnType: 'PgTimestamp', enumValues: undefined, config: [Object], table: [Circular *1], withTimezone: false, precision: undefined, mapFromDriverValue: [Function: mapFromDriverValue], mapToDriverValue: [Function: mapToDriverValue] }, updatedAt: PgTimestamp { name: 'updatedAt', primary: false, notNull: false, default: [SQL], defaultFn: undefined, hasDefault: true, isUnique: false, uniqueName: 'dv_account_updatedAt_unique', uniqueType: undefined, dataType: 'date', columnType: 'PgTimestamp', enumValues: undefined, config: [Object], table: [Circular *1], withTimezone: false, precision: undefined, mapFromDriverValue: [Function: mapFromDriverValue], mapToDriverValue: [Function: mapToDriverValue] } }, [Symbol(drizzle:BaseName)]: 'account', [Symbol(drizzle:IsAlias)]: false, [Symbol(drizzle:ExtraConfigBuilder)]: [Function (anonymous)], [Symbol(drizzle:IsDrizzleTable)]: true, [Symbol(drizzle:PgInlineForeignKeys)]: [ ForeignKey { reference: [Function (anonymous)], onUpdate: 'no action', onDelete: 'cascade', table: [Circular *1] } ] } ```

Additional logging showed that the array of columns for the indexes existed but each element was undefined:

common.ts Line ~175
...
      for (const indexName in indexes) {
        const index = indexes[indexName].build(table);
        dbml.tab(2);

        console.log('Index Info:', indexes[indexName]);
...

Line ~194
...
        if (is(index, PgPrimaryKey) || is(index, MySqlPrimaryKey) || is(index, SQLitePrimaryKey)) {
          console.log('PRIMARY KEY INDEX', index.columns);
          const pkColumns = wrapColumns(index.columns, this.buildQueryConfig.escapeName);
          dbml.insert(`${pkColumns} [pk]`);
        }

...

Here's the output:

Index Info: PrimaryKeyBuilder {
  columns: [ undefined, undefined ],
  name: undefined
}
PRIMARY KEY INDEX [ undefined, undefined ]

This is getting passed to the wrapColumns function and if the columns are undefined then it crashes.

Ultimately I'm not sure if the table data is getting produced by drizzle itself and thus we would have to fix the errors downstream or if this library is generating the data and producing bad column data, or also very possibly, my drizzle config is wrong and its not generating the tables correctly.

In the end, the easiest fix for me was to just have the wrapColumns class check if it was getting undefined columns and then return an empty string if it was. As this is mostly a quick and dirty way for me to convert my drizzle schema's to a close approximation visual representation that seems to work for me, although it certanly looks like the output is sans the [PK] symbols,

I have made a pull request to update the wrap columns to check if it's getting undefined data so that I can at least have it finish its output.

Let me know if you see something obvious I'm missing.

L-Mario564 commented 4 months ago

Appreciate all the info provided. I haven't updated this package to be compatible with Drizzle's new index API, which seems to be the cause for the error here.

Your PR doesn't seem like a definitive solution to this issue, but I'll keep it open in the meantime, in case I need it as reference.

L-Mario564 commented 4 months ago

Can't seem to replicate the issue. Could you provide me the definition for createTable and defaultFields util functions and the users table? The first two are probably the most important, so no issues if you can't share the users table.

johnkraczek commented 4 months ago
import { env } from "~/env";

// DB_PREFIX is just a string that changes based on the environment
// so that development tables don't mix with production tables but I can use the same database

export const createTable = pgTableCreator((name) => `${env.DB_PREFIX}_${name}`);
export const defaultFields = () => {
  return {
    id: text("id").primaryKey().$defaultFn(UID),
    createdAt: timestamp("createdAt", {
      mode: "date",
    }).defaultNow(),
    updatedAt: timestamp("updatedAt", {
      mode: "date",
    }).defaultNow(),
  };
};

I have several other tables that don't have an issue that also use this defaultfields function to get fields I would use on basically every table.

It's only the tables that have extra configs that error out.

Again Thanks for developing this.

At the very least I used my workaround so that it didn't crash and I was able to get an output.

I will watch for when this gets updated with drizzles new index API.

johnkraczek commented 4 months ago
import { boolean, json, text, timestamp } from "drizzle-orm/pg-core";
import { createTable, defaultFields } from "../../utils";
import { relations } from "drizzle-orm";
import { accounts } from "./provider-account";
import { twoFactorMethod } from "./two-factor-methods";
import { token } from "./user-token";
import { userTeam } from "../team/UserTeam";
import { userRole } from "../roles/user-roles";

// =================== USER ROLE OPTIONS ===================
export enum UserRole {
  USER = "USER",
  ADMIN = "ADMIN",
}

// =================== TABLE DEFINITION ===================

export const users = createTable("user", {
  name: text("name"),
  email: text("email").notNull(),
  emailVerified: timestamp("emailVerified", { mode: "date" }),
  image: text("image"),
  hashedPassword: text("hashedPassword").default("").notNull(),
  roles: json("roles").$type<UserRole[]>().default([UserRole.USER]),
  isTwoFactorEnabled: boolean("isTwoFactorEnabled").default(false),
  hasCompletedOnboarding: boolean("hasCopletedOnboarding").default(false),
  ...defaultFields(),
});

// =================== RELATIONSHIPS ===================

export const usersRelations = relations(users, ({ many }) => ({
  accounts: many(accounts),
  twoFactorMethod: many(twoFactorMethod),
  token: many(token),
  userTeams: many(userTeam),
  userRoles: many(userRole),
}));
L-Mario564 commented 4 months ago

I see you're exporting the UserRole Typescript enum. Could you try moving that elsewhere so the when importing * as schema it doesn't go through the rest of the schema stuff?

L-Mario564 commented 2 months ago

@johnkraczek Should be fixed in latest release.