PyWorkflowApp / visual-programming

A Python Visual Programming Workspace for Data Science
MIT License
31 stars 12 forks source link

Node execution with predecessor data #37

Closed reelmatt closed 4 years ago

reelmatt commented 4 years ago

This builds on the read/write of Node data to files @diegostruk1 introduced in PR #35.

The key changes are in pyworkflow/workflow.py and are:

Includes some basic error-checking/exception-handling, but not a lot. You can test execution using sample_execution.zip.

  1. Move join1.csv and join2.csv to /tmp
  2. Load execution_workflow.json into Postman
  3. Manually execute Nodes 1-4. You should see the file names output to screen, and the "Retrieve data" test should return the data written to /tmp.

Edit: latest commits add more error-checking/handling. Namely, endpoints should now return a 400/500 code with message for missing data files/improper execution for most cases.

diegostruk commented 4 years ago

Looks good!