search

scottbishopdev / RoverInterviewProject

A project I completed in about two weeks while interviewing for Rover.com in 2018. This is a re-upload of my original Rover_Interview_Project repo.
GNU General Public License v3.0
0 stars 0 forks source link

Research migrations for MongoDB #11

Open scottbishopdev opened 4 years ago

scottbishopdev commented 4 years ago

I'm admittedly pretty lost when it comes to how we might implement this, but I'm sure some good options exist out there. This issue will be used to track the research process for figuring out how MongoDB migrations should be done, along with what packages or other technologies we should use to accomplish it. Additionally, we should create any necessary additional issues for this project as part of this issue.

Note: We shouldn't begin work on this issue until the project to implement Docker and Docker Compose is completed.

scottbishopdev commented 4 years ago

OK, for starters, we should note that schema migrations and data seeding are two separate concerns. I believe that the reason that this ticket treats them as one is because of my experience with EF Migrations and data seeding in FactorioProductionCells, where the migration framework can be responsible for creating some seed data. In the case of FactorioProductionCells, it populates a few reference tables with data to represent enum objects the application uses, which is in following with Entity Framework's code-first philosophy.

Since we're not using EF here, that combined approach probably shouldn't be used. Currently, I'm researching options for seeding data, and have thus stumbled across mongoose-data-seed and mongo-seeding. I'm currently leaning towards the latter since a docker image is available for it (meaning little or no new code for me), but I'm concerned that I won't have enough control over the process. Both options want the data to be made available in JSON format (though mongo-seeding also allows the user of JavaScript and TypeScript files to create the seed data), which means that some pre-processing to convert the CSV data into JSON would be necessary regardless.

scottbishopdev commented 4 years ago

I'm probably thinking about all this incorrectly. MongoDB and it's docker image supports running a JavaScript-based init script on database startup, which seems like the perfect place to do this kind of data transformation and loading. One potential issue, though, is the limitations that MongoDB's JavaScript interpreter will present. I need to look into whether or not an init script can support things like requiring packages, which I doubt it does...

scottbishopdev commented 4 years ago

I've created #16 to handle the creation of a script to check for the existence of the test data and load it if it's not present.