Open LawJolla opened 2 years ago
Hi @LawJolla, you are almost there I see! Let's get you to the final part 💯
So Prisma Migrate (v2 & v3) relies on a _prisma_migrations
table to record which migrations were applied (with success or not).
Since you are coming from Prisma 1 with an existing database, it doesn't exist
When this table doesn't exist migrate dev
will always ask for reset and is not safe to use against a production database.
You will need to baseline your production database, it will create that table and mark the baseline migration to be ignored.
I would recommend you to use pg_dump
to create the baseline migration to get the perfect copy of your database schema.
Then try it out locally with prisma migrate dev
(no need to baseline locally).
If your local env works, then you can go for base lining the production (or staging) database using prisma migrate resolve [....]
See more details in https://www.prisma.io/docs/guides/database/developing-with-prisma-migrate/baselining
Bash example
# Create `migrations` directory and a `20220323155236_baseline` directory and move there
mkdir -p migrations/$(date +%Y%m%d%H%M%S)_baseline && cd "$_"
# Create a "dump" of the database schema in a `migration.sql` file
pg_dump --file=migration.sql --schema-only --dbname=tests --host=localhost --username=prisma --password
Thanks @Jolg42 !
I've dumped by prod db to migrate. I've been coding against the dumped prod. Now I'm trying to baseline that dumped prod.
The part I'm not understanding is to get a baseline, it wants to reset the database. Is that normal? I just need to do it against a dump.. of the dump? :) And then see if it baselines the original migrate dump?
(I can't baseline the actual prod schema because of all of the 1 to 3 breaking changes. I'll need to take the site down, dump prod, and then try to apply all of the schema changes. I just want to make sure the prod database will be in a condition to migrate.)
Hey @LawJolla, sorry for not following up here sooner. I think read this, was confused a bit and then hoped someone else would follow up.
Could you please explain your steps again 1 by 1, starting with your Prisma 1 database? We really need to understand what exactly you are doing (which commands you run against which database etc) to be able to figure this out.
It should start something like this:
...
to ... and modify ... manually ...Then I think we can understand better. Thanks! Let's get this over the finish line.
Thanks @janpio !
I'll do my best, but I've done the migration over many months didn't write down every command.
I have a huge project (5 servers, 2 front ends) that I've developed over the past 5 years where I'm the only developer. Migrating from Prisma 1 to 2 and Nexus migration required months of work to overcome the technical debt.
My end goal is to migrate my current production schema from Prisma 1 to 3 and merge my Prisma 3 branches into master
. In other words, cut everything over to production data.
What's currently stopping that cut over is I'm unable to get my current migration schema to work with prisma migrate
because of schema incompatibilities.
Let me try this from a high level overview.
pg_dump
ed my database and cloned my production schema and datapg_restore
ed my database so my migration schema and production schema are on the same db instance. (It's low traffic, so while using my production database as a migration test target isn't great practice, it works for my application)prisma2
package for my monoreponpx prisma-upgrade
schema.prisma
fileschema.prisma
file, and the migration database.prisma migrate
(sorry for using "migrate" everywhere 🤣) my migration schema, I get the errors described in the first post and am unsure how to make the schema.prisma
compatible with the current migration schema on my Postgres instance.Did that help at all? 😬
Of course it did.
A few questions:
a) What do you refer to by "migration schema"? A copy of your database that you tried to adapt to be Prisma 2+ compatible?
b) What does "4. I made a prisma2 package for my monorepo" mean exactly?
c) What branch was the work of steps 4 to 5 made? ALso the prisma-3
one you mention later?
d) Where are you trying prisma migrate
on in step 8? What command exactly?
e) How do you imagine the final "update" to work for you? Do you still have all the SQL changes you made to your database along the way manually, triggered by prisma-upgrade
or other realizations?
f) I assume the data in your production database changed a lot since you started this?
Thanks for the response!
Of course it did.
A few questions: a) What do you refer to by "migration schema"? A copy of your database that you tried to adapt to be Prisma 2+ compatible?
Yes. I'm trying to use the Postgres terminology correctly (database vs schema vs table) but I may be wrong. Here's a diagram that will hopefully help.
b) What does "4. I made a prisma2 package for my monorepo" mean exactly?
That I don't know how to write clear sentences. 😬 🙈 I meant to say "I made a prisma2 branch for my monorepo." Sorry!
c) What branch was the work of steps 4 to 5 made? ALso the
prisma-3
one you mention later?
Same branch. Sorry for the confusion. Midway through I changed the target from Prisma 2 to 3.
d) Where are you trying
prisma migrate
on in step 8? What command exactly?
▶ prisma migrate dev --name initial-migration --create-only
e) How do you imagine the final "update" to work for you? Do you still have all the SQL changes you made to your database along the way manually, triggered by
prisma-upgrade
or other realizations?
Yes, I saved the manual SQL commands as I went along. To the extent I'm missing any, I'm sure I can figure them out again. (I cut my teeth in the LAMP stack, so while I'm not a SQL expert, I'm not afraid of it either)
Once I know I can get my migration schema "cc$migration" to work with prisma migrate
, I imagine the final update will involve me taking the site down on a weekend. I'll copy the "wk$prod" Prisma 1 schema as "cc$prod" and apply all of the prisma-upgrade and manual updates I just referenced above. Then I'll merge the prisma3
branch into production and hopefully the change will be complete.
f) I assume the data in your production database changed a lot since you started this? Yes. It's a car dealership with a full admin backend, an advertising server (generates Facebook and other ads), etc.
It's one of the bigger projects you'll see tacked by one developer.
My Prisma 3 schema is 1772 lines.
And here's the types from just part of one of the servers...
All of that is to say I'm, at times, overwhelmed by this migration and greatly, greatly appreciate your and Prisma's support. I've been a part of the community since Graphcool and really went all in on Prisma 1.
And if you're curious, the front end is at:
Thanks again!
What is the state of the _prisma_migrations
tabel before you get to step 8? If it is empty or even missing, I think you really need to baseline and probably misunderstood above.
Let me try to explain again:
You are getting the Drift detected: Your database schema is not in sync with your migration history.
message because your migration history table and folders are empty, but your database already has some tables. That is usually bad.
To avoid that, we have a thing called "baselining" which tells Migrate that this is all expected. You achieve that by a) creating a dump of the database structure into a .sql
file, and b) putting that into a migration folder and telling Migrate that you "already applied it" with e.g. npx prisma migrate resolve --applied 20210426141759_initial-migration-for-db
.
@Jolg42 had shared two commands above that should help with that: https://github.com/prisma/p1-to-p2-upgrade-path-feedback/issues/14#issuecomment-1076475030
When you have done that, the next run of migrate dev
should not show this message any more but probably tell you that your database is up to date with your Prisma schema and there is nothing to do.
Your Prisma 1 stack
What are you building with Prisma?
I am building ... multiple services, including GraphQL and REST, backed by a Prisma/Postgres instance.
Which version of Prisma 1 are you running?
I am running Prisma version ... 1.34
How do you access your Prisma server?
I am accessing my Prisma server with ... Prisma Client
Are you currently using
nexus-prisma
(with Prisma 1)?How many Prisma services are you running on your production Prisma server?
2
Which database (incl. version) do you use?
I am using ... Postgres 10
Where is your Prisma server hosted?
My Prisma server is hosted on ... Fargate
Where is your database hosted?
My database is hosted on ... AWS
Where is your API server hosted?
My API server is hosted on ... Render
Your upgrade plans
What's your preferred upgrade strategy?
Do you want to use the new Nexus Framework when you upgrade?
Y
When do you want to upgrade?
Do you have any remaining questions or comments?
Thank you for everyone's hard work!
I need help syncing my schema.prisma with my migrated Prisma 1 db schema.
For background, I'm a solo developer with a massive Prisma 1 project. I've been working on this project since Prisma was GraphCool 2.0 so the amount of technical debt is enormous.
I spent at least 150 hours over the last several months migrating. I did the following..
Copied the Prisma 1 production schema for migration. Ran the prisma upgrade tool against my Prisma 1 datamodel. Made various SQL migrations manually. Changed Prisma Client 1 calls to Prisma Client 3. Upgraded Nexus (the bulk of my time)
What I'd like to do now it copy my current Prisma 1 schema, run the needed migrations, and cut the updated servers over to the new Prisma 3 client / migrated Prisma 1 schema.
The problem is I'm unable to sync my migration schema with schema.prisma.
Specifically, I can't even get an initial migration. I followed the directions here: https://www.prisma.io/docs/guides/database/developing-with-prisma-migrate/add-prisma-migrate-to-a-project#update-the-development-environment
I do not have any migration folder or files. However, I get the following output. Any help is very appreciated!