Migrate from DynamoDB to RDS

[Douglas-Hong]

Summary

• Migrate all current data from DynamoDB to postgres.
• Use Drizzle as relational ORM.
• Remove support for legacy MongoDB users.
  • This will only affect users that saved their schedules last two years ago, which is fine.

Notes

• Before migrating, set up AWS CLI credentials, make sure you have the correct DynamoDB table name in your .env file, run pnpm run generate to generate the migration files, and make sure the Postgres DB specified in apps/backend/src/db/index.ts exists. Then, run pnpm run migrate to execute the migration.

Migration Procedures

• [ ] Start saving to both DBs.
• [ ] Copy to new DB, while still writing to both.
  • Have to pay attention to timestamp to not override data that came in during migration.
• [ ] Switch read to new DB.
• [ ] Remove old DB.

Action Items

• [x] Auth/user relations.
• [x] Schedules/courses relations.
• [x] Custom events relations.
• [x] Make APIs for new DB.
  • Read/write changes happen here. Don't change API.
• [x] Migration script(s).
  • NOTE: This needs to mangle duplicate schedule names because the (schedule, username) pair is unique.
• [x] Migration procedures
• [x] Provision dev and prod databases.
  • Just one dev database for non-database PRs.
  • PRs that change the DB just dev locally or will be manually provisioned.
• [ ] Check migration and dual-writing locally.
• [x] CI/CD pipelines (#1029)
  • Only migrate when merging into main?
• [ ] Execute migration

Related Issues

• Part of #890

Future Followup

• Google auth #1027
• Only update changed schedule(s) instead of all of them.

[MinhxNguyen7]

I'm going to look at and finish up this PR

[MinhxNguyen7]

Now that I'm actually writing the migration scripts, I've realized that this is more work than I gave it credit for.

I'm going to try to get this done in the next week or so, but I don't like my chances

[MinhxNguyen7]

@ecxyzzy I'm gonna need to work with you on this. We've been on this rodeo together, so it's not too crazy, but we need to review the plan, review the migration script, and help with the deployment.