pytroll / pygac

A python package to read and calibrate NOAA and Metop AVHRR GAC and LAC data
https://pygac.readthedocs.org/
GNU General Public License v3.0
20 stars 27 forks source link

replacing median with 50th percentile #97

Open oliversus opened 3 years ago

oliversus commented 3 years ago

Method correct_times_median should provide median values for year, doy, and milliseconds to correct invalid timestamps. However, np.median will return floats, which could even be fractions due to np.median's internal interpolation method of the two closest values. I'd suggest to use np.percentile instead, where one can explicitly set the interpolation method. Setting this to "nearest" will always return the value closest to the statistical median, thus not changing its type.

np.median([1, 2, 3, 4, 5, 6]) returns 3.5 np.percentile([1, 2, 3, 4, 5, 6], 50, interpolation="nearest") returns 3

This will also avoid a ValueErrorin method to_datetime64, line 449, where conversion to datetime64 is only possible with year as an integer.

codecov[bot] commented 3 years ago

Codecov Report

Merging #97 (07d4de7) into main (a8b68a3) will increase coverage by 0.05%. The diff coverage is 90.90%.

@@            Coverage Diff             @@
##             main      #97      +/-   ##
==========================================
+ Coverage   74.39%   74.44%   +0.05%     
==========================================
  Files          32       32              
  Lines        2882     2888       +6     
==========================================
+ Hits         2144     2150       +6     
  Misses        738      738              
Impacted Files Coverage Δ
pygac/reader.py 81.85% <85.71%> (+0.07%) :arrow_up:
pygac/tests/test_reader.py 98.36% <100.00%> (+0.01%) :arrow_up:
mraspaud commented 1 year ago

@oliversus do you have time to fix this soon?

oliversus commented 1 year ago

@oliversus do you have time to fix this soon?

Hi Martin and Stephan, sorry for ignoring this for so long. I do have a bit of time right now and will try to implement this as Stephan suggested. Cheers Olli