sematic-ai / sematic

An open-source ML pipeline development platform
Other
969 stars 58 forks source link

Add more information about pod before deleting job #1118

Closed augray closed 5 months ago

augray commented 5 months ago

Closes #1116

Make a more robust attempt to collect final pod information and display that in the pods tab before deleting a k8s job.

Testing

Ran a variety of things on the testing pipeline. Here is the final "pods" tab for different scenarios.

Exit with 42 status code

pods_exit_42

Pod OOM

pods_oom

Pod successful exit

pods_successful

Run canceled from UI

pods_delete_from_ui

Manually deleted the job with kubectl (ex: outside Sematic)

pods_deleted_manually