openclimatefix / Satip

Satip contains the code necessary for retrieving, transforming and storing EUMETSAT data
https://satip.readthedocs.io/
MIT License
41 stars 28 forks source link

HRV Coordinate Data variables are mostly NaN #174

Open jacobbieker opened 1 year ago

jacobbieker commented 1 year ago

Describe the bug

For the HRV Zarrs, we save out each timestep's x_geostationary and y_geostationary coordinates to their own data variables, as the HRV image gets moved throughout the day. In at least some of the zarrs on GCP, those data variables are mostly all NaNs. The coordinates are there, based off the first timestep of the year, but the coordinate data variables are mostly NaNs.

Not sure why, as the only work done is copying the values to new data arrays and adding them to the dataset, and they've been kept in float64 the whole time and not compressed either. Luckily its not a lot of data, although if it does need to be recomputed, that would still take awhile to download the datasets, open them, and save out the coordinates. The actual image data seems fine for all of them, so should only really come into effect for ones on the edges of the images.

To Reproduce

Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior

A clear and concise description of what you expected to happen.

Additional context

Add any other context about the problem here.

jacobbieker commented 1 year ago

We could just download a day's worth of data for a few random days throughout the year, but should be able to calculate the updated coordinates for each of them. The times and dates that HRV changes is fairly well documented, so we could calculate what the coordinates are for each timestep and add those.