Check for zeros in the data - Githubissues

openclimatefix / nwp-consumer

Microservice for consuming NWP data.

9 stars 3 forks source link

Check for zeros in the data #159

Open peterdudfield opened 1 week ago

peterdudfield commented 1 week ago

Detailed Description

It would be great to have a check in place that checks for zeros. A large amount of these is normally an error

Context

good to catch data problems early and fail hard

Possible Implementation

add a a check, if number os zeros >20% of the data, error
@devsjc can you suggest a place in the code for this check?

devsjc commented 6 days ago

Yes, I would add it to the filter here: https://github.com/openclimatefix/nwp-consumer/blob/372359ce7789a8da62964bbc333484d2343ee8fa/src/nwp_consumer/internal/service/consumer.py#L410-L424