Earth-2 Model Intercomparison Project (MIP) is a python framework that enables climate researchers and scientists to inter-compare AI models for weather and climate.
source - main @ git commit: ebd86637ec853c11e9ff5e717b053351c740e526
On which installation method(s) does this occur?
Source
Describe the issue
When running 32 ensemble members with 1 rank, the later ensemble members have the perturbation applied multiple times. So the second ensemble member on each rank has a perturbation applied twice. The third ensemble member has a perturbation applied three times. Etc. Is this behavior expected?
This could make the magnitude of the perturbations very large, unless there is some perturbation magnitude normalization applied later.
Using the correlated IC perturbation method with noise amplitude 0.05, the first ensemble member has this magnitude of perturbation:
The 15th ensemble member has this perturbation:
And the 31st ensemble member has this perturbation:
I think there could be two causes:
x is being overwritten by the perturb method here, so as the loop moves forward, the perturbed x gets perturbed again, instead of the original x getting perturbed.
Version
source - main @ git commit: ebd86637ec853c11e9ff5e717b053351c740e526
On which installation method(s) does this occur?
Source
Describe the issue
When running 32 ensemble members with 1 rank, the later ensemble members have the perturbation applied multiple times. So the second ensemble member on each rank has a perturbation applied twice. The third ensemble member has a perturbation applied three times. Etc. Is this behavior expected?
This could make the magnitude of the perturbations very large, unless there is some perturbation magnitude normalization applied later.
Using the correlated IC perturbation method with noise amplitude 0.05, the first ensemble member has this magnitude of perturbation:![image](https://github.com/NVIDIA/earth2mip/assets/15190307/ddeb71d8-4811-4742-8fd4-987799d0b0f8)
The 15th ensemble member has this perturbation:![image](https://github.com/NVIDIA/earth2mip/assets/15190307/8489da74-1c81-4b45-ab7c-be0e5e7fafcd)
And the 31st ensemble member has this perturbation:![image](https://github.com/NVIDIA/earth2mip/assets/15190307/32d507d0-365a-41e0-b27b-0d89c7c6f88c)
I think there could be two causes:
Environment details
No response