search

mozilla / ssh_scan_api

An API for ssh_scan (https://github.com/mozilla/ssh_scan) and the backend API service for the Mozilla SSH Observatory (https://observatory.mozilla.org/)
31 stars 12 forks source link

Attempt a dry run of a production data from mongodb to a local postgres DB #126

Closed claudijd closed 6 years ago

claudijd commented 6 years ago

This is a task to connect to the existing production db, which is mongo, and run a migration using mongo2postgres.rb. If there are any bugs fix them, do some manual verification's to make sure everything still works as expected.

claudijd commented 6 years ago

Dry run was attempted today and although there was a small percentage that didn't migrate in my test due to unicode being present in the target field, I think the results are reasonably promising to proceed with the migration using the script with some minor updates, which I'll push shortly.

claudijd commented 6 years ago

Also, to give an expectation for how long the migration will take, we're in the magnitude of < 1min (when performed via SSH tunnel), we'll be doing the actual migration locally, so it will likely be faster than that.