sematic-ai / sematic

An open-source ML pipeline development platform
Other
975 stars 59 forks source link

Add apis for blocking on a run and getting its output by run id #1103

Closed augray closed 1 year ago

augray commented 1 year ago

Closes #161

Prior to this PR, we had the ability to get an artifact using the artifact id, but the artifact id was a bit tricky to get programmatically. Usually the script that launched a run only had access to the run's id. This PR adds the ability to get the value based on the run id. This is a bit tricky without knowing that the run has actually completed though, so I also add an api for blocking until the run has terminated.

Testing

bazel run //sematic/examples/testing_pipeline:__main__ -- --silent
bazel run //sematic/examples/testing_pipeline:__main__ -- --silent --block
bazel run //sematic/examples/testing_pipeline:__main__ -- --block
bazel run //sematic/examples/testing_pipeline:__main__ -- --block --detach --cloud

Confirmed that the outputs were as expected.