Implement Workflows in COAL-SDS

[lewismc]

For full end-to-end automation, we need to eventually implement OODT's Workflow Management capabilities.

[lewismc]

@alexwyi19 @ahuerta33 @DenimMazuki @kristencheung can you please assign yourselves to whatever issue we had previous discussed? Thanks

[lewismc]

./crawler_launcher --printSupportedCrawlerActions

https://github.com/capstone-coal/coal-sds/wiki/Automation-of-Data-Acquisition-and-Crawling Flags to add are

-wm,    --workflowMgrUrl <url> which is most likely http://localhost:9001

Additionally you need to add a post ingestion actionId

e.g. 
  Action:
    Id: TriggerPostIngestWorkflow
    Description: Triggers workflow event with the name [ProductType]Ingest
    Phases: [postIngestSuccess]

This should appended to

--actionIds DeleteDataFile

[lewismc]

The workflow documentation for OODT resides at http://oodt.apache.org/site_docs/cas-workflow/ you can see the bottom left=hand-side for the navigation links to http://oodt.apache.org/site_docs/cas-workflow/development/developer.html http://oodt.apache.org/site_docs/cas-workflow/user/basic.html http://oodt.apache.org/site_docs/cas-workflow/user/advanced.html I would read them through thoroughly and then begin working within the workflow component for coal-sds.

[lewismc]

I've committed a lot of code to cover this issue e.g. https://github.com/capstone-coal/coal-sds/commit/fc1418de9085e397ff6b4fe1a474d3b09801cfa8 There is more work to do be to automate the initiation of the workflows directly after the crawler_launcher has successfully ingested the AVIRIS products. Additionally, the new workflow policy needs to be documented such that workflows are easily understood. This will however be covered in https://github.com/capstone-coal/coal-sds/issues/13