search

rcos / rcos-data

RCOS database definition and internal API.
MIT License
3 stars 1 forks source link

Migrate old data #8

Open rushsteve1 opened 3 years ago

rushsteve1 commented 3 years ago

The old versions of Observatory and the WIki need to be migrated to the new database as soon as we have a new production DB running.

Old data can be found on:

Apexal commented 3 years ago

Postgrest's bulk insert routes will come in handy here.

rushsteve1 commented 3 years ago

Probably the easiest and best way to do this would be to transfer the old DO Droplet to the new RCOS account, then pull the data off of it. This would also preserve a backup of the old server in case we ever need it.

Actually migrating the data might be a bit of a pain though. We're probably going to have to use mongoexport or mongodump to dump the data to JSON (or CSV?), and then edit it to match the new schema before importing it to Postgres. Something tells me this is going to be harder than it seems though.

rushsteve1 commented 3 years ago

Attacked is a ZIP containing a script to dump files from BSON to JSON, and then a set of jq filters to transform the JSON to a more compatible schema.

data-transforms.zip