openclimatefix / power_perceiver

Machine learning experiments using the Perceiver IO model to forecast the electricity system (starting with solar)
MIT License
7 stars 1 forks source link

Enlarge topography data so we never get NaNs over land #90

Open JackKelly opened 2 years ago

JackKelly commented 2 years ago

we definitely got NaNs in the topo data. But maybe that was over ocean!

maybe it's already big enough

AssertionError:

In [1]: %debug
> /home/jack/dev/ocf/power_perceiver/power_perceiver/np_batch_processor/topography.py(223)_get_surface_height_for_satellite()
    221         surface_height_for_batch[example_idx] = aligned["surface_height"].values
    222 
--> 223     assert np.isfinite(surface_height_for_batch).all()
    224 
    225     return surface_height_for_batch

ipdb> np.isfinite(surface_height_for_batch).all()
False
ipdb> np.isfinite(surface_height_for_batch).shape
(8, 128, 256)
ipdb> np.isfinite(surface_height_for_batch).all(axis=1)
array([[ True,  True,  True, ...,  True,  True,  True],
       [ True,  True,  True, ...,  True,  True,  True],
       [ True,  True,  True, ...,  True,  True,  True],
       ...,
       [ True,  True,  True, ...,  True,  True,  True],
       [ True,  True,  True, ...,  True,  True,  True],
       [False, False, False, ..., False, False, False]])
ipdb> satellite.isel(example=-1)
<xarray.DataArray (y: 128, x: 256)>
array([[-791502.  , -790220.6 , -788939.56, ..., -476123.28, -474908.5 ,
        -473693.94],
       [-791033.25, -789752.1 , -788471.3 , ..., -475706.4 , -474491.8 ,
        -473277.44],
       [-790565.  , -789284.1 , -788003.56, ..., -475289.97, -474075.53,
        -472861.3 ],
       ...,
       [-736504.7 , -735250.1 , -733995.9 , ..., -426936.03, -425740.84,
        -424545.88],
       [-736091.5 , -734837.1 , -733583.06, ..., -426564.5 , -425369.47,
        -424174.62],
       [-735678.7 , -734424.5 , -733170.6 , ..., -426193.28, -424998.4 ,
        -423803.72]], dtype=float32)
Coordinates:
    y_geostationary  (y) float32 4.472e+06 4.471e+06 ... 4.346e+06 4.345e+06
    x_geostationary  (x) float32 -1.804e+06 -1.803e+06 ... -1.55e+06 -1.549e+06
Dimensions without coordinates: y, x
ipdb> quit