Netflix / metaflow

Open Source Platform for developing, scaling and deploying serious ML, AI, and data science systems
https://metaflow.org
Apache License 2.0
8.2k stars 771 forks source link

Expose more accurate task / run lifecycle via Client API #1512

Open tuulos opened 1 year ago

tuulos commented 1 year ago

Today, you can use the Client API to see whether a task/run has completed successfully (including caught exceptions) through .finished and .successful attributes.

Through the current API, you are not able to see the state of runs that

It would be useful to expose more accurate task/run state information through the Client API, some of which is already visible in the Metaflow UI (through heartbeats).

jixianyihao commented 10 months ago

This feature we really need, like spark connect server, https://spark.apache.org/docs/latest/spark-connect-overview.html spark provide API make IDE or Jupyterlab can submit job use API; It is very important。