ProjectSidewalk / SidewalkWebpage

Project Sidewalk web page
http://projectsidewalk.org
MIT License
84 stars 25 forks source link

Dev env improvement when importing users: merge instead of delete and replace #3721

Open misaugstad opened 3 weeks ago

misaugstad commented 3 weeks ago
Brief description of problem/feature

I added a new import-users.sh script to the dev env that will wipe the data in your sidewalk_login schema and replace it with the data from the supplied dump. If you want to add db dumps for multiple cities to your dev env, and the new one you want to add is a more recent dump than what you currently have, you'd need to run the script to import a more up-to-date users list.

The downside is that it may break the your dev env for the old db dump for the same reason that you needed to import it in the first place! If you've been testing stuff in your dev env, you're likely making new accounts. These are new accounts that aren't in the new users dump that you're importing; your list of users has essentially diverged from the list of users on prod.

The way to get around this right now is, whenever you import the db for a new city, you'd need to re-import the dbs for all cities that you already have in your dev env to wipe out any local-only user accounts.

Potential solution(s)

A better solution would be to have the import-users.sh script actually merge the list of users you're importing with the list of users that you currently have. If an imported user already exists, then you skip it. If it doesn't, add it. No need to delete the whole thing.

Deleting the whole thing and starting fresh should still be an option though as a command line parameter, and maybe a total wipe is even the default?

Could be a fun task for someone interested in working on database stuff!

misaugstad commented 3 weeks ago

The other issue with fully deleting the sidewalk_login schema is that the process actually removes all the foreign key constraints that reference that schema. It doesn't really cause an issue, but we want our dev environments to match prod as closely as possible, so we'd like to keep those constraints in place, which we could do by merging.