search

coursera / dataduct

DataPipeline for humans.
Other
252 stars 83 forks source link

The create-load-redshift step should run with stage set to false #167

Closed warhammerkid closed 8 years ago

warhammerkid commented 8 years ago

Currently this step downloads all input files to the local server, even though it does nothing with them. This is because stage is set to 'true', which implies that it needs the input files locally (see the ShellCommandActivity docs for details).

There is currently a pull request to fix this issue, but it does it by not passing in the input node, rather than correctly setting stage to false.

cliu587 commented 8 years ago

I've made a PR for this: https://github.com/coursera/dataduct/pull/179

Thanks for the note!

sb2nov commented 8 years ago

Fixed in #179