search

coursera / dataduct

DataPipeline for humans.
Other
252 stars 83 forks source link

How to create ShellCommandActivity #266

Open JeremyColton opened 7 years ago

JeremyColton commented 7 years ago

Hi, I saw in your docs to use:

step_type:transform
name: check_inactive
script: check_inactive.sh
no_output: true

This creates the following nodes:

  1. S3DataNode - check_inactive.S3Node0
  2. S3DataNode - TransformStep1.S3Node0
  3. ShellCommandActivity - check_inactive.ShellCommandActivity0
  4. ShellCommandActivity - TransofrmStep1.ShellCommandActivity0

I only expect and want the 3rd node to be created. Why are the others created? How do I config my node in dataduct to remove these obsolete nodes?