PEDSnet / pedsnetdcc

CLI tool for PEDSnet data coordinating center ETL tasks
Other
0 stars 0 forks source link

Automating migration of id_maps schemas #75

Closed murphyke closed 7 years ago

murphyke commented 8 years ago

pedsnetdcc should learn how to migrate the id map schemas (I know Charlie would prefer me to say schemata) prior to transformation into pedsnetdcc.

E.g. either a sub-command:

pedsnetdcc copy_map_schemas --from DBURI --to DBURI where the DBURIs can refer to the same or different database clusters but must refer to different databases in any case.

Or a flag on the transform sub-command:

pedsnetdcc [--maps=DBURI] transform ...

In the latter case, --maps could be a mandatory flag that would also accept arguments of reuse and new.

The specification of schema names now becomes an issue; at the very least, we'd need to specify target schema names.

Although the transform code does not currently make any assumption about schema names, it would be simpler and maybe safer for the user if we institutionalized the schema names we've been using. If we do that for all the schema names, we can replace the search path flag with a --site=SITE flag for all commands and just automatically synthesize the right search path ....

The infomodels commands would still require search paths, unless they were wrapped by pedsnetdcc.

gracebrownecodes commented 8 years ago

Look at https://github.research.chop.edu/dbhi/pedsnet/issues/114 for legacy practices.

eceowl commented 7 years ago

Automate Creation of id_map schemas and tables

murphyke commented 7 years ago

@eceowl I don't know if you intend that trello link to be accessible to us. It's fine if it's just a private scratchpad for yourself.

eceowl commented 7 years ago

Fixed.