search

skypilot-org / skypilot

SkyPilot: Run LLMs, AI, and Batch jobs on any cloud. Get maximum savings, highest GPU availability, and managed execution—all with a simple interface.
https://skypilot.readthedocs.io
Apache License 2.0
6.19k stars 425 forks source link

[Logging] Symlink latest logs to a `latest` path #3617

Open romilbhardwaj opened 1 month ago

romilbhardwaj commented 1 month ago

We should have a symlinked path like ~/sky_logs/latest/ which links to the logs of the last run.

We see a lot of questions around how to debug provisioning and see logs, and having a symlinked path in addition to the usual timestamped path would make that easier.

Similar to how ray has a session_latest dir under /tmp/ray/

-rw-r--r--   1 romilb  wheel    98 May 30 10:03 prom_metrics_service_discovery.json
drwxrwxrwx   8 romilb  wheel   256 May 30 10:00 session_2024-05-30_10-00-19_878676_78762
drwxrwxrwx  10 romilb  wheel   320 May 30 10:02 session_2024-05-30_10-02-54_265418_80116
lrwxr-xr-x   1 romilb  wheel    49 May 30 10:02 session_latest -> /tmp/ray/session_2024-05-30_10-02-54_265418_80116
ksasi commented 1 week ago

I think that the creation of symlink can happen just for cluster creation and deletion (local_up and local_down) as most of the questions may be for these operations. Looking for your thoughts and suggestions.