rstoneback commented 2 years ago

Description

Addresses #3, #15, #28, #29

A large range of changes to get pysatSeasons generally up to the modern era.

Switched to workflows and pytest
Condensed testing
Added Consellation support to more functions, with testing
Added xarray support to functions, with testing
Removed deprecated library use
Improved docstrings and general style
Adopted updated setup.py and setup.cfg
Removed collections.deque use
computational_form -> to_xarray_dataset
Simplified code for seasonal averaging functions
Updated examples

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

Existing unit tests
Developed additional unit tests
Ran updated documentation examples

Test Configuration:

Operating system: MacOS
Version number: Python 3.9
Any details about your local setup that are relevant: pysat 'develop' branch and v3.0.1 release.

Checklist:

[x] Update the version number
[x] Verify and update Zenodo information in .zenodo.json
[x] doc tests pass locally
[x] activate this branch on readthedocs (https://pysatseasons.readthedocs.io/en/rcv0-2-0/)
[x] Upload the test distribution to the testPyPi server
[x] Contacted developers and users about the future release
[x] Performed integration tests with downstream packages (include list of these packages)
[x] Create a wiki page with the release text (https://github.com/pysat/pysat/wiki/pysatSeasonsv0.2.0-Release-Text)
[x] After approval, confirm all changes from the RC pull are listed in the wiki release text.
[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules
[x] Add a note to CHANGELOG.md, summarizing the changes

rstoneback commented 2 years ago

Getting the COSMIC and IVM example to work again is going to require changes to the way cosmic data is loaded and binned. xarray is not handling electron profiles with different lengths.

rstoneback commented 2 years ago

COSMCI profiles work again. Changes here to get both xarray and pandas working currently interferes with simple numpy scalar array returns, when appropriate.

rstoneback commented 2 years ago

COSMCI profiles work again. Changes here to get both xarray and pandas working currently interferes with simple numpy scalar array returns, when appropriate.

Correction, the issue is only for xarray 1D sources.

rstoneback commented 2 years ago

COSMIC IVM example runs and plots for me during a short 2 day test window. I am currently running the full 67 day date range as in the example.

rstoneback commented 2 years ago

I tried to upload to test pypi but it says I am not allowed to upload to pysatSeasons. I can confirm my user is not listed as a maintainer on the test site. I am listed on the full pypi.

I've uploaded the code here as a placeholder: pysatSeasons-0.2.0rc1.tar.gz

rstoneback commented 2 years ago

For reasons I don't understand the auto API isn't building correctly online, but it does build correctly locally.... https://pysatseasons.readthedocs.io/en/rcv0-2-0/api.html

aburrell commented 2 years ago

Looks like there's a camelCase variable in pysatSeasons.occur_prob.daily2D(const, bin1, label1, bin2, label2, data_label, gate, returnBins=False) that should be deprecated and replaced with a snake_case variable name.

rstoneback commented 2 years ago

Fixed returnBins and returnData. GitHub tests aren't running with the push. I'm guessing system issues 🤷

rstoneback commented 2 years ago

Tests pass locally but I wrote the deprecation unit tests using testing functions that haven't been released (yet) in pysat.

rstoneback commented 2 years ago

I put April 15th as a nominal date. More to the point the deadline day for releases is either the 25th (program date), or from a practical standpoint April 29th. The close date for changes to the heliopython environment for the pyhc summer school is May 1.

jklenzing commented 2 years ago

Jeff Review TODO:

[x] test out demo codes
[x] test docstring examples

jklenzing commented 2 years ago

cosmic_and_ivm_demo.py is not running for me. The variable graph is undefined as of line 331, which means something is wrong up top since graph is only conditionally defined.

rstoneback commented 2 years ago

I will double check it. It was converted to the xarray cosmic Instrument.... which is a bit ahead still, release-wise.

jklenzing commented 2 years ago

Apparently I don't have cosmic data for the required range on this computer, which probably would create a problem.

jklenzing commented 2 years ago

The cosmic data center site is down, which means I can't test the code.

rstoneback commented 2 years ago

I added a note/error about pysatCDAAC version requirements for the demo.

jklenzing commented 2 years ago

Pinging @rstoneback

Is this ready to go? Does it need another review?

rstoneback commented 2 years ago

hmm........ we broke something

mkdir pysatData
[3](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:3)
  python -c "import pysat; pysat.params['data_dirs'] = 'pysatData'"
[4](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:4)
  shell: /usr/bin/bash -e {0}
[5](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:5)
  env:
[6](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:6)
    pythonLocation: /opt/hostedtoolcache/Python/3.8.13/x64
[7](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:7)
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.13/x64/lib
[8](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:9)
Traceback (most recent call last):
[9](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:10)
  File "<string>", line 1, in <module>
[10](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:11)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/pysat/__init__.py", line 49, in <module>
[11](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:12)
    from pysat import _params
[12](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:13)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/pysat/_params.py", line 13, in <module>
[13](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:14)
    import pysat.utils
[14](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:15)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/pysat/utils/__init__.py", line 9, in <module>
[15](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:16)
    from pysat.utils._core import available_instruments
[16](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:17)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/pysat/utils/_core.py", line 15, in <module>
[17](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:18)
    import xarray as xr
[18](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:19)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/xarray/__init__.py", line 1, in <module>
[19](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:20)
    from . import testing, tutorial
[20](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:21)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/xarray/testing.py", line 9, in <module>
[21](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:22)
    from xarray.core import duck_array_ops, formatting, utils
[22](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:23)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/xarray/core/duck_array_ops.py", line 26, in <module>
[23](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:24)
    from . import dask_array_compat, dask_array_ops, dtypes, npcompat, nputils
[24](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:25)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/xarray/core/npcompat.py", line 72, in <module>
[25](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:26)
    _SupportsDType[np.dtype],
[26](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:27)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/typing.py", line 261, in inner
[27](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:28)
    return func(*args, **kwds)
[28](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:29)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/typing.py", line 897, in __class_getitem__
[29](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:30)
    _check_generic(cls, params)
[30](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:31)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/typing.py", line 211, in _check_generic
[31](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:32)
    raise TypeError(f"{cls} is not a generic class")
[32](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:33)
TypeError: <class 'numpy.typing._dtype_like._SupportsDType'> is not a generic class
[33](https://github.com/pysat/pysatSeasons/runs/7635355776?check_suite_focus=true#step:7:34)
Error: Process completed with exit code 1.

rstoneback commented 2 years ago

Tests are passing with latest pysat.

rstoneback commented 2 years ago

check if matplotlib dependence is real

rstoneback commented 2 years ago

Tests stopped working due to some kind of numpy and xarray issue. Not knowing what is going on, I bumped the numpy version up to 1.21.... https://github.com/pysat/pysatSeasons/runs/8124117985?check_suite_focus=true

How is it that working software always breaks just before release? 🤷

rstoneback commented 2 years ago

Now a lone windows test failure from matplotlib with the suggestion that perhaps Tcl wasn't installed properly.

pysat / pysatSeasons

Rcv0 2 0 #30

Description

Type of change

How Has This Been Tested?

Checklist: