mikro-orm / mikro-orm

TypeScript ORM for Node.js based on Data Mapper, Unit of Work and Identity Map patterns. Supports MongoDB, MySQL, MariaDB, MS SQL Server, PostgreSQL and SQLite/libSQL databases.
https://mikro-orm.io
MIT License
7.84k stars 548 forks source link

UpsertMany with composite keys coming from foreign entity generates wrong SQL #6252

Open artsiommiksiuk opened 6 days ago

artsiommiksiuk commented 6 days ago

Describe the bug

UpsertMany with composite keys coming from foreign entity generates wrong SQL.

This is cleanedup generated sql from reproduction issue:

INSERT INTO "sample-stick-well"
            (
                        "created_at",
                        "sample_id",
                        "sample_organization_id",
                        "stick_well_id",
                        "updated_at"
            )
            VALUES
            (
                        '2024-11-21T20:33:12.567Z',
                        '9a3b008e-a4f6-4cf6-8568-b65861497432',
                        'f10adaed-6789-47a3-8c6b-d9906441e94b',
                        'text12314334',
                        '2024-11-21T20:33:12.567Z'
            )
            ,
            (
                        '2024-11-21T20:33:12.567Z',
                        '117449c9-6abb-41c4-a52a-6428054816e1',
                        'f10adaed-6789-47a3-8c6b-d9906441e94b',
                        'text98989847',
                        '2024-11-21T20:33:12.567Z'
            )
on conflict
            (
                        "organization",
                        "id"
            )
            do UPDATE
set    "sample_organization_id" = excluded."sample_organization_id",
       "created_at" = excluded."created_at",
       "updated_at" = excluded."updated_at",
       "stick_well_id" = excluded."stick_well_id"

Looks like generated on conflict section must be:

on conflict
(
            "sample_organization_id",
            "sample_id"
)

Instead of

on conflict
(
            "organization",
            "id"
)

Workaround I've found, which proving my guess, is manually setting the onConflictFields to correct values works.

await this.orm.em.upsertMany(SampleStickWellEntity, entities, {
    // @ts-expect-error
    onConflictFields: ["sample_id", "sample_organization_id"]
});

Reproduction

The setup is quite lengthy. Sorry for that.

@Entity({ tableName: "organization" })
export class OrganizationEntity {
    @PrimaryKey({ type: "uuid" })
    id: string;

    @Property()
    name: string;
}

@Entity({ tableName: "sample" })
export class SampleEntity {    
    @ManyToOne(() => OrganizationEntity, { primary: true })
    organization: Ref<OrganizationEntity>;
    @PrimaryKey({ type: "uuid" })
    id: string;
    [PrimaryKeyProp]!: ["organization", "id"];

    ...
}

@Entity({ tableName: "sample-stick-well" })
@Index<SampleStickWellEntity>({ properties: ["sample", "createdAt", "updatedAt", "stickWellId"]})
export class SampleStickWellEntity {

    @ManyToOne(() => SampleEntity, { primary: true })
    sample: Ref<SampleEntity>;
    [PrimaryKeyProp]!: ["sample"];

    @CreatedAt()
    createdAt: Date;
    @UpdatedAt()
    updatedAt: Date;

    @Property()
    stickWellId: string;
}

This should be the call itself.

await this.orm.em.upsertMany(SampleStickWellEntity, [{ 
    sample: ref(SampleEntity, ["id1", "id2"]), 
    stickWellId: "value", 
    createdAt: new Date(), 
    updatedAt: new Date()
}]);

Error:

err: {
      "type": "InvalidFieldNameException",
      "message": "insert into \"sample-stick-well\" (\"created_at\", \"sample_id\", \"sample_organization_id\", \"stick_well_id\", \"updated_at\") values ('2024-11-22T07:57:23.199Z', '117449c9-6abb-41c4-a52a-6428054816e1', 'f10adaed-6789-47a3-8c6b-d9906441e94b', 'text98989847', '2024-11-22T07:57:23.199Z'), ('2024-11-22T07:57:23.199Z', '9a3b008e-a4f6-4cf6-8568-b65861497432', 'f10adaed-6789-47a3-8c6b-d9906441e94b', 'text12314334', '2024-11-22T07:57:23.199Z') on conflict (\"organization\", \"id\") do update set \"sample_organization_id\" = excluded.\"sample_organization_id\", \"created_at\" = excluded.\"created_at\", \"updated_at\" = excluded.\"updated_at\", \"stick_well_id\" = excluded.\"stick_well_id\" - column \"organization\" does not exist",
      "stack":
          InvalidFieldNameException: insert into "sample-stick-well" ("created_at", "sample_id", "sample_organization_id", "stick_well_id", "updated_at") values ('2024-11-22T07:57:23.199Z', '117449c9-6abb-41c4-a52a-6428054816e1', 'f10adaed-6789-47a3-8c6b-d9906441e94b', 'text98989847', '2024-11-22T07:57:23.199Z'), ('2024-11-22T07:57:23.199Z', '9a3b008e-a4f6-4cf6-8568-b65861497432', 'f10adaed-6789-47a3-8c6b-d9906441e94b', 'text12314334', '2024-11-22T07:57:23.199Z') on conflict ("organization", "id") do update set "sample_organization_id" = excluded."sample_organization_id", "created_at" = excluded."created_at", "updated_at" = excluded."updated_at", "stick_well_id" = excluded."stick_well_id" - column "organization" does not exist
              at PostgreSqlExceptionConverter.convertException (/Users/user1/Projects/backend/node_modules/.pnpm/@mikro-orm+postgresql@6.2.0_@mikro-orm+core@6.2.0/node_modules/@mikro-orm/postgresql/PostgreSqlExceptionConverter.js:42:24)
              at PostgreSqlDriver.convertException (/Users/user1/Projects/backend/node_modules/.pnpm/@mikro-orm+core@6.2.0/node_modules/@mikro-orm/core/drivers/DatabaseDriver.js:344:54)
              at /Users/user1/Projects/backend/node_modules/.pnpm/@mikro-orm+core@6.2.0/node_modules/@mikro-orm/core/drivers/DatabaseDriver.js:348:24
              at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
              at async SqlEntityManager.upsertMany (/Users/user1/Projects/backend/node_modules/.pnpm/@mikro-orm+core@6.2.0/node_modules/@mikro-orm/core/EntityManager.js:818:21)
              at SampleStickWellService.upsert (/Users/user1/Projects/backend/apps/be/src/domain/sample-stick-well.service.ts:41:9)
              at <anonymous> (/Users/user1/Projects/backend/apps/be/src/routes/qrmap.ts:131:9)
              at <anonymous> (/Users/user1/Projects/backend/apps/be/src/utils/express-oas2.ts:145:13)
              at <anonymous> (/Users/user1/Projects/backend/apps/be/src/packages/express-async-handler/index.ts:9:16)

              at Parser.parseErrorMessage (/Users/user1/Projects/backend/node_modules/.pnpm/pg-protocol@1.6.1/node_modules/pg-protocol/src/parser.ts:369:69)
              at Parser.handlePacket (/Users/user1/Projects/backend/node_modules/.pnpm/pg-protocol@1.6.1/node_modules/pg-protocol/src/parser.ts:188:21)
              at Parser.parse (/Users/user1/Projects/backend/node_modules/.pnpm/pg-protocol@1.6.1/node_modules/pg-protocol/src/parser.ts:103:30)
              at Socket.<anonymous> (/Users/user1/Projects/backend/node_modules/.pnpm/pg-protocol@1.6.1/node_modules/pg-protocol/src/index.ts:7:48)
              at Socket.emit (node:events:518:28)
              at addChunk (node:internal/streams/readable:559:12)
              at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)
              at Readable.push (node:internal/streams/readable:390:5)
              at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
              at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
      "code": "42703",
      "length": 113,
      "name": "InvalidFieldNameException",
      "severity": "ERROR",
      "position": "444",
      "file": "parse_relation.c",
      "line": "3648",
      "routine": "errorMissingColumn"
    }

What driver are you using?

@mikro-orm/postgresql

MikroORM version

6.2.0

Node.js version

20.11.1

Operating system

MacOS

Validations

B4nan commented 5 days ago

6.2.0

Please always try latest version before you report anything.

Also, I'd appreciate a complete reproduction instead of those snippets.

B4nan commented 3 days ago

Not reproducible on my end, here is a passing reproduction based on what you provided, let's reopen with a failing one.

import {
  Entity,
  ManyToOne,
  MikroORM,
  PrimaryKey,
  PrimaryKeyProp,
  Property,
  ref,
  Ref,
} from '@mikro-orm/postgresql';

@Entity()
class Organization {

  @PrimaryKey({ type: 'uuid' })
  id!: string;

  @Property()
  name!: string;

}

@Entity()
class Sample {

  @ManyToOne(() => Organization, { primary: true })
  organization!: Ref<Organization>;

  @PrimaryKey({ type: 'uuid' })
  id!: string;

  [PrimaryKeyProp]?: ['organization', 'id'];

}

@Entity({ tableName: 'sample-stick-well' })
class SampleStickWell {

  @ManyToOne(() => Sample, { primary: true })
  sample!: Ref<Sample>;

  [PrimaryKeyProp]?: 'sample';

  @Property()
  createdAt = new Date();

  @Property()
  updatedAt = new Date();

  @Property()
  stickWellId!: string;

}

let orm: MikroORM;

beforeAll(async () => {
  orm = await MikroORM.init({
    dbName: '6252',
    entities: [SampleStickWell],
    debug: true,
  });
  await orm.schema.refreshDatabase();
});

afterAll(async () => {
  await orm.close(true);
});

test('6252', async () => {
  const oid = 'ac21172f-98b0-436f-aeda-923d5e61748f';
  const sid = '4953caed-cb20-4cb4-939e-958b1874a090';

  await orm.em.insert(Organization, { id: oid, name: 'org' });
  await orm.em.insert(Sample, { id: sid, organization: oid });

  await orm.em.upsertMany(SampleStickWell, [{
    sample: ref(Sample, [oid, sid]),
    stickWellId: 'value',
    createdAt: new Date(),
    updatedAt: new Date(),
  }]);
});
artsiommiksiuk commented 2 days ago

@B4nan, sorry for the initial poor description. I've managed to make a reproducible based on your attempt with latest (6.4.0) version.

Turned out, that created as class instances entities are not working, while simple objects are fine.

import {
  Entity,
  ManyToOne,
  MikroORM,
  PrimaryKey,
  PrimaryKeyProp,
  Property,
  ref,
  type Ref,
} from '@mikro-orm/postgresql';
import test, { after, before } from 'node:test';

@Entity()
export class Organization {
  @PrimaryKey({ type: 'uuid' })
  id!: string;

  @Property()
  name!: string;
}

@Entity()
export class Sample {
  @ManyToOne(() => Organization, { primary: true })
  organization!: Ref<Organization>;

  @PrimaryKey({ type: 'uuid' })
  id!: string;

  [PrimaryKeyProp]?: ['organization', 'id'];

}

@Entity()
export class SampleStickWell {
  @ManyToOne(() => Sample, { primary: true })
  sample!: Ref<Sample>;
  [PrimaryKeyProp]?: ['sample'];

  @Property()
  createdAt = new Date();

  @Property()
  updatedAt = new Date();

  @Property()
  stickWellId!: string;

}

let orm: MikroORM;
const oid = 'ac21172f-98b0-436f-aeda-923d5e61748f';
const sid = '4953caed-cb20-4cb4-939e-958b1874a090';

before(async () => {
  orm = await MikroORM.init({
    dbName: '6252',
    entities: [SampleStickWell],
    debug: true,
    allowGlobalContext: true,
  });
  await orm.schema.refreshDatabase();

  await orm.em.insert(Organization, { id: oid, name: 'org' });
  await orm.em.insert(Sample, { id: sid, organization: oid });
});

after(async () => {
  await orm.close(true);
});

// not working
test('6252 class constructor', async () => {
  const ssw = new SampleStickWell();
  ssw.sample = ref(Sample, [oid, sid]);
  ssw.stickWellId = "value";
  ssw.createdAt = new Date();
  ssw.updatedAt = new Date();

  await orm.em.upsertMany(SampleStickWell, [ssw]);
});

// working
test('6252 object literal', async () => {
  await orm.em.upsertMany(SampleStickWell, [{
    sample: ref(Sample, [oid, sid]),
    stickWellId: 'value',
    createdAt: new Date(),
    updatedAt: new Date(),
  }]);
});
B4nan commented 2 days ago

Thanks, will take another look later today.