lvxhnat / ion

A fully integrated platform for aggregating, visualising and analysing alternative data
GNU General Public License v3.0
12 stars 1 forks source link

feat [ion-clients]: use Alembic for data migration #116

Open wardzxzxc opened 1 year ago

wardzxzxc commented 1 year ago

Currently models are populated via SQLAlchemy which works for now. However, there will be issues later on when we try to change the schemas of the DB. The naive way is to drop all data, recreate the new schemas via SQLAlchemy, etc, but this process will purge all data stored. The other way is to manually create the ALTER statements for the table and patch the current data to suit the new schemas. This is very error-prone and cumbersome. Suggestion is to migrate to Alembic for data migration and state handling.

Alembic: https://alembic.sqlalchemy.org/en/latest/

Using SQLAlchemy vs Alembic: https://stackoverflow.com/questions/30425214/what-is-the-difference-between-creating-db-tables-using-alembic-and-defining-mod