search

openclimatefix / graph_weather

PyTorch implementation of Ryan Keisler's 2022 "Forecasting Global Weather with Graph Neural Networks" paper (https://arxiv.org/abs/2202.07575)
MIT License
197 stars 50 forks source link

Error while running the `run_full.py` for the gfs-reforecast dataset from hugging face #74

Closed SwordSaintLancelot closed 8 months ago

SwordSaintLancelot commented 1 year ago

Describe the bug

While running and debugging the python file mentioned, I found that the data is sliced for only one time stamp whereas in the next lines it is mentioned that the data has a starting and ending time. After reading through the paper, my understanding is that one node has one timestamp and model is trained only be of one timestamp but that contradicts the start and end variables in the XrDataset dataloader. image

I do have another question regarding the aux_dims in the encoder. If I run the example usage, it shows the issue that output and input dimensions do not match as input feature dimensions sum up to a total of 102 (input_dims(78) + aux_dim(24)) whereas the output dims are just 78. image

To Reproduce

Steps to reproduce the behavior:

  1. Go to hf_forecasts.json.
  2. Change the first file name to data/forecasts/GFSv16/2021/04/2021040100.zarr.zip as other files are not found on the webpage
  3. Go to train/run_full.py
  4. Run the file.
  5. See error

Steps to reproduce the behavior:

  1. Go to train.
  2. Create a new file for example usage.
  3. Run the file.
  4. See error

Expected behavior

These steps should give you the errors as shown in the images.

Additional context

Also, I could not find the files mentioned in the hf_forecasts.json on hugging face gfs-reforecast dataset.

jjjj32481 commented 1 year ago

Replacing the line 35, i.e., "data = self.data.isel(reftime=item, time=slice(start_idx, start_idx + 1))", in XrDataset definition with the following line may be effective. data = self.data.isel(reftime=item, time=slice(start_idx, start_idx + 2))

SwordSaintLancelot commented 1 year ago

Thank you for the response. I have already implemented this line and have tested running the model and it works. I just wanted to make sure, if this indeed is an error, it might be better to correct this in the main repo as well.

peterdudfield commented 11 months ago

Thanks @jjjj32481 and @SwordSaintLancelot would you be able to do a PR?

@jjjj32481 do you know why this is?

SwordSaintLancelot commented 11 months ago

Surely, It's a small adjustment. I can make the PR for this.

SwordSaintLancelot commented 11 months ago

@peterdudfield I don't think I have write access to the repository. Should I still look into creating the PR?

peterdudfield commented 11 months ago

Yea, I think you can fork the repo, and then make a PR back to this one?