skypilot-org / skypilot

SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.
https://skypilot.readthedocs.io
Apache License 2.0
6.54k stars 467 forks source link

[Storage]Misleading storage information displayed with sky storage ls with identical storage names in different clouds #2030

Closed landscapepainter closed 7 months ago

landscapepainter commented 1 year ago

Currently, when storages from different cloud providers, say gcp and cloudflare r2, are created with identical storage names, some information in state.db are overwritten due to the following script ran in global_user_state.py/add_or_update_storage().

   _DB.cursor.execute('INSERT OR REPLACE INTO storage VALUES (?, ?, ?, ?, ?)',
                       (storage_name, storage_launched_at, handle, last_use,
                        storage_status.value))

And this results into misleading information from what sky storage ls displays and perhaps other bugs in the future. Say there's a sky managed storage called train-data from gcs, if we create another sky managed storage called train-data from r2, the data related to the train-data from gcs in storage table from state.db will be replaced. And this is due to having storage_name set as the PRIMARY KEY.

Consider the following scenario:

sky storage ls 

NAME                              CREATED      STORE   COMMAND                                                                         STATUS         
train-data                     36 mins ago   GCS   sky launch test.yaml --cloud gcp -y                                        READY          

sky launch another_test.yaml --cloud gcp -y # creates a storage called train-data from R2
sky storage ls 

NAME                              CREATED      STORE   COMMAND                                                                         STATUS         
train-data                      2 mins ago  GCS, R2  sky launch another_test.yaml --cloud gcp -y                      READY 

Need to figure out a way to separate storages from different clouds with identical storage_name in state.db to fix this issue.

github-actions[bot] commented 11 months ago

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

github-actions[bot] commented 7 months ago

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

github-actions[bot] commented 7 months ago

This issue was closed because it has been stalled for 10 days with no activity.

romilbhardwaj commented 7 months ago

Closed by #2063.