Imagenet requires a larger disk space than the RTX6000 GPU nodes have

teaching-on-testbeds / ml-energy

MIT License

0 stars 0 forks source link

Imagenet requires a larger disk space than the RTX6000 GPU nodes have #6

Open ffund opened 6 months ago

ffund commented 6 months ago

A few options:

create a shared filesystem (see this artifact). You will include instructions for creating the share, and a separate notebook for reserving a node that is connected to a storage network and mounts this share.
alternative: check (via the Hardware Browser) if there is a node with GPU and enough storage. If you can get a reservation on that node.
alternative: object store

Deepak-Work commented 6 months ago

Tested mounting shares on a node on Monday with success. Downloading data on the share worked as well.

Today the mounting gives the following error:

Trying to figure out the issue now.

Deepak-Work commented 5 months ago

The above issue still persists. Could not find a way to resolve this issue. This method worked for me once. Once crashed, I have never been able to get it to work. Trying to get time on hardware with more resources does not seem possible. I could not find them to be ever free. Still exploring object store.