[FEATURE] Overriding partitionKey Prefix in Single Table Design

gmreburn commented 3 months ago

Summary:

I am struggling to implement multiple inheritance for classes while overriding the partitionKey's prefix in a single table design. The base table should define the partition key (pk) without any prefix, and each derived class should override it to add a specific prefix/suffix. I couldn't find any relevant guidance in the documentation. The alternative of defining each class without inheritance seems cumbersome.

Any suggestions on how to achieve this?

Code sample:

export default class Table extends Entity {
  // @attribute.partitionKey.string()
  pk: string;
}
export default class UserModel extends Table {
  @attribute.partitionKey.string({ prefix: "USER" })
  pk: string;
}

[ValidationError]: Attribute "pk" should be decorated in "n" Entity.
// Uncomment the partionKey in Table class, you get this error:
[DynamodeStorageError]: Attribute "pk" was already decorated in entity "d"

Other:

[x] I have read through the Dynamode documentation before posting this issue
[x] I have searched through the GitHub issues (including closed issues) and pull requests to ensure this feature has not already been suggested before

gmreburn commented 3 months ago

Note that I am running in Next.js. This seems to work in dev mode but not when building. Maybe this is an issue with Next.js build processing.

blazejkustra commented 3 months ago

Hey @gmreburn! Could you provide a minimal reproduction repo?

[ValidationError]: Attribute "pk" should be decorated in "n" Entity. // Uncomment the partionKey in Table class, you get this error:

Looking at the error, I think it is connected to the fact that next minifies the code and your class name was changed which is breaking the underlying logic 😅

blazejkustra commented 3 months ago

Also to answer your question, yes this is the correct way to add prefixes to your entities 😄

gmreburn commented 3 months ago

Hey, thanks for reply. I added serverMinification: false to my next.config which got me past some of these errors. I removed the @attribute.partitionKey.string({ prefix: "USER" }) and will revisit this later on. There is probably a better way but that works for now :)

Here is my config in case others run into this:

/**
 * @type {import('next').NextConfig}
 */
const nextConfig = {
  experimental: {
    serverMinification: false,
  },
};

module.exports = nextConfig;

It might also be nice to add a name attribute to this library for the entity's name to bypass this minification issue.

@blazejkustra , are you still interested in a minimal reproduction repo?

blazejkustra commented 3 months ago

If it's not too much work you can create a minimal reproduction repo 😄

Another way to fix your issue could be to use this flag (I haven't tested it).

It might also be nice to add a name attribute to this library for the entity's name to bypass this minification issue.

Let's keep this issue open, I'll think about this

gmreburn commented 3 months ago

I was unable to reproduce the original issue that I reported when making a new repository but I ran into another issue. I pushed the changes to https://github.com/gmreburn/next-dynamode/tree/dynamode-issue-32 because I expected the prefix "USER#" to be applied to the pk. You can view this PR to see the changes I applied - https://github.com/gmreburn/next-dynamode/pull/1/files

I've seen the prefix apply in my other repository so maybe I missed something 🤷

The steps to reproduce are in the readme.md

I'm not sure if renaming properties within the class is supported. I renamed pk to id in the User class. Since I wasn't sure whether this is supported, I also tried without renaming the column. This had no impact to prefix being applied. Please let me know if you see anything.

blazejkustra commented 3 months ago

I'll have a look this week 👀

blazejkustra commented 3 months ago

@gmreburn There are several issues in the model you created. After these changes it should work properly!

Before:

import Table, { tableManager, TableProps } from "./table";
import attribute from "dynamode/decorators";

export interface UserProps extends Omit<TableProps, "pk"> { // ⚠️ Different props than the table, you have to extend the original props so that table constructor matches user constructor
  id: string;
}

export class User extends Table {
  @attribute.partitionKey.string({ prefix: "USER#" }) // ⚠️ prefix defined with a separator (separator is added automatically)
  id: string; // ⚠️ two partition keys, one named pk the other id (it's not supported)

  constructor(props: UserProps) {
    super({
      pk: props.id,
    });

    this.id = props.id;
  }
}

export const UserManager = tableManager.entityManager(); // ⚠️ User was not passed here so you were using a wrong manager

After:

import Table, { tableManager, TableProps } from "./table";
import attribute from "dynamode/decorators";

export interface UserProps extends TableProps {}

export class User extends Table {
  @attribute.partitionKey.string({ prefix: "USER" })
  pk!: string;

  constructor(props: UserProps) {
    super(props);
  }
}

export const UserManager = tableManager.entityManager(User);

blazejkustra commented 3 months ago

Here is an example of production code, how I imagine modeling entities with dynamode:


import attribute from 'dynamode/decorators';
import Entity from 'dynamode/entity';
import TableManager from 'dynamode/table';

import '../../utils/aws';

export type UserTablePrimaryKey = {
  pk: string;
  sk: string;
};

export type UserTableProps = UserTablePrimaryKey & {
  createdAt?: Date;
  updatedAt?: Date;

  gsi_pk_2?: string;
  gsi_sk_2?: string;

  gsi_pk_3?: string;
  gsi_sk_3?: string;
};

export const USER_TABLE_NAME = process.env.USER_TABLE_NAME || 'user-development';
export const DYNAMODE_INDEX = 'dynamode-index';
export const GSI_2_INDEX = 'GSI_2_INDEX';
export const GSI_3_INDEX = 'GSI_3_INDEX';

export default class UserTable extends Entity {
  @attribute.partitionKey.string()
  pk: string;

  @attribute.sortKey.string()
  sk: string;

  @attribute.gsi.partitionKey.string({ indexName: DYNAMODE_INDEX })
  dynamodeEntity!: string;

  @attribute.gsi.sortKey.string({ indexName: DYNAMODE_INDEX })
  gsi_sk_1: string;

  @attribute.gsi.partitionKey.string({ indexName: GSI_2_INDEX })
  gsi_pk_2?: string;

  @attribute.gsi.sortKey.string({ indexName: GSI_2_INDEX })
  gsi_sk_2?: string;

  @attribute.gsi.partitionKey.string({ indexName: GSI_3_INDEX })
  gsi_pk_3?: string;

  @attribute.gsi.sortKey.string({ indexName: GSI_3_INDEX })
  gsi_sk_3?: string;

  @attribute.date.string()
  createdAt: Date;

  @attribute.date.string()
  updatedAt: Date;

  constructor(props: UserTableProps) {
    super(props);

    this.pk = props.pk;
    this.sk = props.sk;
    this.createdAt = props.createdAt || new Date();
    this.updatedAt = props.updatedAt || new Date();
    this.gsi_sk_1 = this.createdAt.toISOString();
    this.gsi_pk_2 = props.gsi_pk_2;
    this.gsi_sk_2 = props.gsi_sk_2;
    this.gsi_pk_3 = props.gsi_pk_3;
    this.gsi_sk_3 = props.gsi_sk_3;
  }
}

export const UserTableManager = new TableManager(UserTable, {
  tableName: USER_TABLE_NAME,
  partitionKey: 'pk',
  sortKey: 'sk',
  indexes: {
    [DYNAMODE_INDEX]: {
      partitionKey: 'dynamodeEntity',
      sortKey: 'gsi_sk_1',
    },
    [GSI_2_INDEX]: {
      partitionKey: 'gsi_pk_2',
      sortKey: 'gsi_sk_2',
    },
    [GSI_3_INDEX]: {
      partitionKey: 'gsi_pk_3',
      sortKey: 'gsi_sk_3',
    },
  },
  createdAt: 'createdAt',
  updatedAt: 'updatedAt',
});

import attribute from 'dynamode/decorators';

import UserTable, {
  GSI_2_INDEX,
  GSI_3_INDEX,
  UserTableManager,
  UserTablePrimaryKey,
  UserTableProps,
} from './UserTable';

type UserProps = UserTableProps & {
  email: string;
  isVerified: boolean;
  username?: string;
};

export default class User extends UserTable {
  // pk -> userId
  // sk -> userId
  @attribute.string()
  userId: string;

  @attribute.string()
  email: string;

  @attribute.boolean()
  isVerified: boolean;

  @attribute.string()
  username?: string;

  @attribute.gsi.partitionKey.string({ indexName: GSI_2_INDEX, prefix: User.name })
  gsi_pk_2: string;

  @attribute.gsi.sortKey.string({ indexName: GSI_2_INDEX })
  gsi_sk_2: string;

  @attribute.gsi.partitionKey.string({ indexName: GSI_3_INDEX, prefix: User.name })
  gsi_pk_3?: string;

  @attribute.gsi.sortKey.string({ indexName: GSI_3_INDEX })
  gsi_sk_3: string;

  constructor(props: UserProps) {
    super(props);

    this.email = props.email;
    this.isVerified = props.isVerified;
    this.username = props.username;
    this.userId = props.pk;

    this.gsi_pk_2 = props.email;
    this.gsi_sk_2 = this.createdAt.toISOString();

    this.gsi_pk_3 = props.username;
    this.gsi_sk_3 = this.createdAt.toISOString();
  }

  static getPrimaryKey(userId: string): UserTablePrimaryKey {
    return {
      pk: userId,
      sk: userId,
    };
  }
}

export const UserManager = UserTableManager.entityManager(User);

const userId = 'userId';
const email = 'email';
new User({ ...User.getPrimaryKey(userId), email, isVerified: false });

blazejkustra commented 3 months ago

It might also be nice to add a name attribute to this library for the entity's name to bypass this minification issue.

Implemented in https://github.com/blazejkustra/dynamode/pull/33.

Steps to test the above PR:

First install it with: pnpm i dynamode@1.4.1-rc.3

Use this code:


import Table, { tableManager, TableProps } from "./table";
import attribute, { entity } from "dynamode/decorators";

export interface UserProps extends Omit<TableProps, "pk"> { pk: string; }

@entity.customName("USER_ENTITY") export class User extends Table { @attribute.partitionKey.string({ prefix: "USER" }) pk!: string;

constructor(props: UserProps) { super(props); } }

export const UserManager = tableManager.entityManager(User);


- Run `pnpm build`

The outcome is that you get entity names even with minification:
![image](https://github.com/user-attachments/assets/80aed15b-584a-4205-a337-a8c2e7cd4e42)

blazejkustra commented 3 months ago

Let me know if it makes sense and works for you @gmreburn

gmreburn commented 3 months ago

This is great info, thank you for sharing! I see some nice hints/tips in the examples provided. I will take a look at using dynamode@1.4.1-rc.3 with named entities. I think this will help get past my build issue.

blazejkustra commented 3 months ago

Let me know if custom names for entities work as expected, once you confirm this I'll release it in v1.5.0 😄 Also consider starting this repo if you like dynamode ⭐

gmreburn commented 3 months ago

Yes, this worked! I was able to remove serverMinification: false, from my next.config.js file (which was added as a quick fix to bypass the issue caused by Next.js minification) after installing 1.4.1-rc.3 and decorating the models. Thank you!

blazejkustra commented 3 months ago

Released in v1.5.0 🚀

blazejkustra / dynamode