search

WPRDC / wprdc-etl

MIT License
8 stars 3 forks source link

removed config file handling #56

Closed saylorsd closed 8 years ago

saylorsd commented 8 years ago

fixes #55

moved connecting ahead of duplicate checking in the pipeline workflow which prevents double connecting

turned status loggin off by default

saylorsd commented 8 years ago

@bsmithgall The CLI command used a config file which I haven't accounted for yet, so this won't pass the build test. I was thinking of parsing the config file within the scripts code and then sending them as individual arguments to the pipeline. Maybe along the lines of moving the pipeline.set_config_from_file() code to pipeline.scripts and then writing a new function in pipeline to handle a dict of arbitrary settings. Or.... now that I type this all out, maybe it would make more sense to keep a lot of the old config file stuff in pipeline, but rework it a bit so that using a file is optional.