search

microbiomedata / nmdc-runtime

Runtime system for NMDC data management and orchestration
https://microbiomedata.github.io/nmdc-runtime/
Other
7 stars 3 forks source link

[Parent] Make nmdc-runtime development environment setup easier #755

Open PeopleMakeCulture opened 2 weeks ago

PeopleMakeCulture commented 2 weeks ago

@eecavanna I am working on simplifying the dev setup for nmdc-runtime by including a step in make up-dev that checks your local mongo, and if it is not populated (or stale?), automatically running a mongodump and restore job. One current blocker is that the names of the mongo backup files are not predictable. They have the format of dump_nmdc-prod_2024-11-06_20-12-02. Is the time portion of the datetime required? If not, could we truncate it to only the date? That way I can programatically grab the most recent mongo backup.

eecavanna commented 2 weeks ago

Hi @PeopleMakeCulture, yes, the time portion is necessary because we sometimes generate multiple backups on the same date (e.g. during the release process). We use the same dumper script in both scenarios.

Also, I don't want to make having NERSC access be a requirement for setting up a development environment.

I wonder if spelling out the sequence of commands one could use to set up their local database would be sufficient (e.g. do X to obtain a dump, then run Y to load it into your database).