search

WPRDC / wprdc-etl

MIT License
8 stars 3 forks source link

added jobs link to readme #53

Closed saylorsd closed 8 years ago

bsmithgall commented 8 years ago
  1. Did you figure out a name?
  2. Can you also fix the link to the builds?
  3. Nice blog post!
bsmithgall commented 8 years ago

:+1: LGTM. We do actually need a name for it though, for pip/installation reasons. When you type pip install <name>, what do you want it to be?

saylorsd commented 8 years ago
  1. Not yet. We've got a whiteboard full of them, but haven't committed yet. We're trying to give it a Pittsburgh flavor in addition to the data transmission/pipe thing. Something with 'bridge' in it is looking pretty good ATM.
  2. Fixed - good looking out!
  3. Thanks!

Also, later this week I'm going to look into refactoring this to just have the pipeline code - I'll get a name by then (I'm a bit of a commitment-phobe) . I've found a few instances were it would be nice to just use the pipeline functionality in other scripts that we have (for one-off data uploads/ repairs as well as in a future data dictionary creation tool). I'm thinking of moving the scripts and status stuff to another repo that would encompass the entire automated ETL setup (pretty much a place for jobs and settings and the code for status recording and the CLI) which will probably require another name, lol. That stuff would have this library as requirement and would go from there.

bsmithgall commented 8 years ago

That's not a bad idea -- if you do that, they should be all have related names.