SolarArbiter / solarforecastarbiter-core

Core data gathering, validation, processing, and reporting package for the Solar Forecast Arbiter
https://solarforecastarbiter-core.readthedocs.io
MIT License
33 stars 21 forks source link

reports should always report number of missing observations #742

Open wholmgren opened 2 years ago

wholmgren commented 2 years ago

Reports can only report missing observations if those observations were uploaded with nan values. Missing values due to discontinuous observations are not caught by discard before resample procedure. _calc_discard_before_resample might need to resample to the observation frequency (not the forecast frequency!) or compute a difference with an expected index to determine the number of missing values. Or maybe that happens in _resample_obs since it has knowledge of the metadata and we probably don't want to mess up the bools returned by _calc_discard_before_resample.

Reports also only report the number of missing resampled observations in combination with a quality flag. Would be good to report missing resampled observations on their own.