Dependencies and datasets

NVIDIA / modulus-makani

Massively parallel training of machine-learning based weather and climate models

Other

227 stars 33 forks source link

Dependencies and datasets #1

Open imangohari1 opened 10 months ago

imangohari1 commented 10 months ago

Hi there, I was wondering if:

Could you add a setup.cfg or similar for dependencies? Using the nvcr.io/nvidia/pytorch:23.09-py3 container , running into AttributeError: module 'torch._C' has no attribute '_cuda_setDevice'
There is no --config="sfno_linear_73chq" in the config file here. Shouldn't that be --config="sfno_linear_73chq_sc3_layers8_edim384_asgl2" instead here?
Could you add some link/download script for hdf5 data that can be used for basic training?

bonevbs commented 10 months ago

Hi @imangohari1 thank you for pointing these out. A setup.cfg will be added soon. There are currently some issues in PyTorch DDP with complex-valued parameters, and we are in the process of releasing a patch. I expect that a setup.cfg will follow as well.

bonevbs commented 10 months ago

You are absolutely right about the config. I have fixed that. The first error is most likely caused due to a CPU-only installation of PyTorch.

imangohari1 commented 10 months ago

Hi @bonevbs Thanks. Do you happen to have considered something for downloading a test set of hdf5 files for training too?