bradyrx / esmtools

:octopus: a toolbox for Earth System Model analysis :octopus:
https://esmtools.readthedocs.io/en/latest/
MIT License
27 stars 4 forks source link

Daskify Stats #70

Closed bradyrx closed 4 years ago

bradyrx commented 4 years ago

Description

Cleans up the stats module using simply numpy functions and adds explicit dask support with testing.

Fixes #66, #65, #31, #63

Type of change

Please delete options that are not relevant.

To Do List

How Has This Been Tested?

Checklist (while developing)

Pre-Merge Checklist (final steps)

References

Please add any references to manuscripts, textbooks, etc.

bradyrx commented 4 years ago

This example from Ryan could potentially help: https://nbviewer.jupyter.org/gist/rabernat/30e7b747f0e3583b5b776e4093266114

aaronspring commented 4 years ago

This looks massive and great. Looking forward to review this once it is ready.

bradyrx commented 4 years ago

I'll tag you to review once it's done! Just doing this in surges when I have energy. But should be really robust and have all the dask support we need.

bradyrx commented 4 years ago

See here: https://github.com/scipy/scipy/blob/47ffc1e/scipy/stats/stats.py#L2767-L2882

They call it nan_policy

bradyrx commented 4 years ago

from statsmodels acf : ['none', 'raise', 'conservative', 'drop']

bradyrx commented 4 years ago

Went ahead and merged @aaronspring, since you reviewed this extensively.