search

payloadcms / payload

Payload is the open-source, fullstack Next.js framework, giving you instant backend superpowers. Get a full TypeScript backend and admin panel instantly. Use Payload as a headless CMS or for building powerful applications.
https://payloadcms.com
MIT License
22.85k stars 1.41k forks source link

Postgres v2 to v3 migration script issues #6630

Closed tomparkp closed 1 month ago

tomparkp commented 2 months ago

Link to reproduction

No response

Payload Version

3.0.0-beta.39

Node Version

18

Next.js Version

15rc

Describe the Bug

I recently upgraded to 3.0.0-beta.39 following the instructions in the release notes but ran into a few issues after the fact.

It's possible I'm misunderstanding something so forgive me if any of this is incorrect.

Per the instructions:

  1. I ran migrate:create before upgrading to beta.39 to verify that I had a current snapshot, this indicated no schema changes were found meaning my latest snapshot was up-to-date.
  2. I removed the dev migration batch.
  3. I upgraded the packages to beta.39 and generated the predefined migration: 
    payload migrate:create --file @payloadcms/db-postgres/relationships-v2-v3
  4. I ran payload migrate

All was well until I went to generate my next migration.

From what I can see the issue is that Payload uses the .json snapshots to diff schema differences and generate the relevant SQL migration logic. However, the relationships-v2-v3 script modifies the database without creating any kind of "post-v3 script" json snapshot.

So upon running migrate:create again we end up with another snapshot and series of SQL changes, however this migration fails because it's attempting to drop tables that were already dropped by the v2-v3 migration script:

image image

Reproduction Steps

  1. Follow the v2-v3 db upgrade instructions.
  2. Run migrate:create after completing all the steps
  3. Observe that the new migration is attempting to perform db changes already made by the previous script.

Adapters and Plugins

No response

tomparkp commented 2 months ago

One other thing I wanted to note - the v2 to v3 script also does not have a down function in the migration, which breaks commands like payload migrate:reset since it modifies the db without providing a rollback.

DanRibbens commented 2 months ago

This is great feedback, this all makes sense.

In one iteration I had made the json file spit out when running the predefined snapshot. I think I can add that back in.

One other thing I wanted to note - the v2 to v3 script also does not have a down function in the migration, which breaks commands like payload migrate:reset since it modifies the db without providing a rollback.

This might be something we can manage, but I'm not sure. Working through the migration up was a huge lift in itself, let alone the schema changes and everything else that change required. If anyone in the community can do this, otherwise this upgrade will stay a one-way change.

tomparkp commented 2 months ago

@DanRibbens Thanks! I ended up solving this by renaming the .json file from the next migration created to match the name of the migration generated for the v2-v3 script and deleted the extra .ts migration, which allows me to create migrations normally again.

As you said I think the creation command for the v2-v3 should create a json snapshot just like the normal migrate:create command, which would keep you guys from getting hit with questions from people running into this when v3 releases.

DanRibbens commented 1 month ago

The main issue here was fixed in https://github.com/payloadcms/payload/commit/ff82bb5661156392e128ee82f21bf0472588ce66.

I don't see any other interest in the down function so I'm going to close this as completed.

Thanks!