Refactor: Making migrations work again.

[mattprintz]

• Renames migrate to migrations.
• Fixes Makefile for proper migration generation
• No longer need separate migration image, reuse api image for migrations
• Fix circular imports in TDS
• Make data seeding to be a migration (Runs once on new setups)
• Reconnected alembic to pydantic for auto migration generation
• Squashed existing migrations
• Created migration to fix pre-existing mismatches between the code and the DB where appropriate
• Added migration to reindex the datasets es index to resolve issue with conflicting index types

[brandomr]

Looks good based on a cursory review--what's the best / most appropriate way to test it?

[mattprintz]

Looks good based on a cursory review--what's the best / most appropriate way to test it?

@brandomr This is intended to be a seamless refactor, so mostly things should just work. If you run it for the first time on an existing codebase with proper volumes, 2 migrations should run, one to correct things that were broken in the code/db syncing before the refactor, and one with the remapping of the datasets index.

To test the from-scratch migration flow, you can delete your TDS volumes and start up and it should run 4 migrations. The squashed setup, the data seeding, the fix for thing that changed, and the dataset remap.

Either way, the next time you start up, it should check for new migrations and not do anything and end pretty quickly.

The default in the docker-compose for seeding has been switched to true, since it's expected that if you're running docker-compose you want some data pre-loaded, but it will only run once since the seeding is done by a migration now, and migrations only run once.

[mattprintz]

These changes look good to me! I resolved your one question @mattprintz. The only tiny tiny nit question is why the Enums were broken up in squash2. My question doesn't matter too much since we're likely phasing out Postgres.

The alembic portion of that migration was all auto-generated, so the movement of the code wasn't intentional. It should generate the same DB state, though, and the source of truth is elsewhere, so I'm not sure if it should matter, but let me know if it does matter.