Closed josef-pkt closed 5 years ago
josef-pkt
commented
7 years ago @ChadFulton I don't know what's new and what has been backported to 0.8 in statespace We need a description for the release notes of the refactoring of the statespace framework and date/time index handling in tsa. Are there new features/models that are not in 0.8?
I will compile a list of the main changes that need to go into the release notes for 0.9.
josef-pkt
commented
7 years ago starting the overview list of main new features and refactorings (may be incomplete or incorrect)
(put the above in a TOC like highlights list in the release notes?)
performance
other enhancements/refactorings by Kerby ?
buglist (I don't know how complete this is) https://github.com/statsmodels/statsmodels/issues?q=is%3Aissue+is%3Aclosed+milestone%3A0.9
ChadFulton
commented
7 years ago I'll get back to this as soon as I can (I am moving to a new house today, so things are a little disrupted right now :)
josef-pkt
commented
7 years ago target for release is in a month, so no rush yet (My family is still shopping for new car and house)
jreback
commented
6 years ago @josef-pkt pinging you here. We would like to remove some last vestigies of code in pandas that statsmodels is currently using in the released versions (though IIRC its removed from master).
WidePanel: https://github.com/pandas-dev/pandas/pull/18341get_dummies (https://github.com/pandas-dev/pandas/issues/18452, but fixedhttps://github.com/statsmodels/statsmodels/commit/45e55d0180017fffd1c57e2b92143c6be0379d4d#diff-a494553e88130577065e0b012246ca51)pandas.core.datetools https://github.com/pandas-dev/pandas/pull/19119any thoughts on a release timetime? (sorry about this, I know I hate when folks ask this question).
cc @TomAugspurger cc @bashtage
jreback
commented
6 years ago side note that @njsmith released patsy 0.5.0 as well, hopefully should have some updated Categorial support.
jreback
commented
6 years ago @josef-pkt any timing on 0.9.0 release?
jreback
commented
6 years ago any update?
josef-pkt
commented
6 years ago I will try for next week. I finished the last 'bug-wrong', and have two other bugs on my short list. There are one to three deprecation warnings in the test suite that will need checking, one from matplotlib and one or two either from numpy or pandas, AFAIR.
jreback
commented
6 years ago thanks!
josef-pkt
commented
6 years ago list of recent ENH
bug-wrong
MAINT
better handling of corner cases e.g. #4087 power if effect size is zero
The above is preliminary, based on going through the last 9 pages of closed PRs (merges between June 24 2017 and April 4 2018)
It does not include the large number of PRs with BUG and DOC fixes.
josef-pkt
commented
6 years ago my list of PRs for 0.9 is missing some PRs that were merged before 0.8 release but not included in 0.8, e.g.
... ?
another problem: github doesn't allow sorting PRs by merge date, the default is sorting by creation date.
older (by creation date) merged ENH PRs :
josef-pkt
commented
6 years ago current short list for what's remaining
open: VAR parts that were broken by VECM quickfix: add intercept back in as attribute, but leave exog alone. (half a day on Monday ?), renamings in VECM and maybe some smaller things that fit into half a day of checking
(build: do I need newer cython for sdist for python > 3.4?)
jreback
commented
6 years ago you should drop 3.4 support FYI (as numpy and pandas have)
josef-pkt
commented
6 years ago I'm currently still using 3.4 as my main python. (After the release I will spend some time on updating my development setup on my computer. "If it works, then don't update more often than every 3 years." Having to use Microsoft compilers when 3.5 was released sounded way to dangerous to me.)
josef-pkt
commented
6 years ago github search query for merged PRs limited by merge date
is:pr is:closed merged:>2018-04-01
this is useful given that github doesn't allow sorting by merge date
maintenance for labeling: merged PRs that have no milestone
is:pr is:closed merged:>2017-02-01 no:milestone
is empty for 0.9, except for PRs merged before 0.8 final release but not backported
josef-pkt
commented
6 years ago a background info
excluding merge commits, squashing varied quite a bit for larger PRs (e.g. Evgeny count models versus Aleksandar VECM) start hash is around the time of 0.8 release and does not contain large changes by Chad made before that
>git shortlog -s -n --no-merges 13016cd8cf6f1608f40b2f1bc6b692e11308e6d6..HEAD
161 Josef
135 Kerby Shedden
99 Kevin Sheppard
67 Evgeny Zhurko
55 vegcev
53 thequackdaddy
45 Chad Fulton
42 Brock Mendel
42 Yichuan Liu
40 Aleksandar Karakas
30 tvanzyl
25 Josef Perktold
12 Jacob Kimmel
5 Christian Lorentzen
5 Pamphile ROY
5 Richard Höchenberger
4 Jamie Morton
4 Tom Augspurger
4 evgenyzhurko
3 Gerard Brunick
3 jbrockmendel
3 malickf
2 Dror Atariah
2 Greg Mosby
2 Jeroen Van Goey
2 Jorge C. Leitao
2 Juan Escamilla
2 Michael Kaminsky
2 Natasha
2 Nick DeRobertis
2 Niels Wouda
2 jarvmiller
2 tommyod
1 Alex Fortin
1 Alexander Belopolsky
1 Dave Willmer
1 Jasmine Mou
1 Jim Correia
1 Joon Ro
1 Jordan Yoder
1 Joses W. Ho
1 José Lopez
1 Justin Bois
1 Kerby A Shedden
1 Kernc
1 Maxim Uvarov
1 Mosky
1 Rob Klooster
1 Roman Ring
1 Scott Tsai
1 Soren Fuglede Jorgensen
1 Tony Jiang
1 U-GELBERGROUP\jengelman
1 ftemme
1 hugovk
1 kiwirob
1 weizhongg
1 zveryanskybuild problems, and changes for it are in comments in #3791
a check for sdist in an old python 3.4 (3 years old which I had available to check the install from sdist) pip install compiled with MingW/gcc works without problems tests run successfully with 3 failures/errors, all minor (or buggy cvxopt)
no problems in statespace (with disabled test_simulate) sdist (with cythonizing) was created in python 3.6 with scipy 1.0.1, and it works with minimal compatible (AFAIU) scipy 0.16.1
================================== FAILURES ===================================
_____________________________ test_getframe_smoke _____________________________
C:\Users\josef\Downloads\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\statsmodels\multivariate\tests\test_factor.py:227: in test_getframe_smoke
from pandas.io import formats as pd_formats
E ImportError: cannot import name 'formats'
During handling of the above exception, another exception occurred:
C:\...\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\statsmodels\multivariate\tests\test_factor.py:229: in test_getframe_smoke
from pandas import formats as pd_formats
E ImportError: cannot import name 'formats'
________________________________ test_testers _________________________________
C:\...\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\statsmodels\stats\tests\test_knockoff.py:81: in test_testers
RegressionFDR(y, x, tv, design_method=method)
C:\...\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\statsmodels\stats\_knockoff.py:83: in __init__
exog1, exog2, _ = _design_knockoff_sdp(exog)
C:\...\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\statsmodels\stats\_knockoff.py:161: in _design_knockoff_sdp
sol = solvers.sdp(c, G0, h0, [G1], [h1])
C:\...\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\cvxopt\coneprog.py:4129: in sdp
= ds)
C:\...\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\cvxopt\coneprog.py:1396: in conelp
misc.update_scaling(W, lmbda, ds, dz)
C:\...\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\cvxopt\misc.py:614: in update_scaling
offsetU = ind2, offsetVt = ind2)
E ArithmeticError: 49
________________________ TestHoltWinters.test_forecast ________________________
C:\...\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\statsmodels\tsa\tests\test_holtwinters.py:125: in test_forecast
[60.9542,36.8505,46.1628,50.1272], 3)
C:\...\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\numpy\testing\utils.py:474: in assert_almost_equal
return assert_array_almost_equal(actual, desired, decimal, err_msg)
C:\...s\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\numpy\testing\utils.py:842: in assert_array_almost_equal
precision=decimal)
C:\...\WinPython-64bit-3.4.3.6\python-3.4.3.amd64\lib\site-pac
kages\numpy\testing\utils.py:665: in assert_array_compare
raise AssertionError(msg)
E AssertionError:
E Arrays are not almost equal to 3 decimals
E
E (mismatch 100.0%)
E x: array([ 60.949, 36.786, 46.082, 50.053])
E y: array([ 60.954, 36.85 , 46.163, 50.127])
= 3 failed, 6289 passed, 63 skipped, 2 xfailed, 44 warnings in 832.72 seconds =>>> pd.__version__
'0.17.0'
>>> scipy.__version__
'0.16.1'
>>> numpy.__version__
'1.9.3'
bashtage
commented
5 years ago Closed, outdated.
a first long list of merge commits in master, to get a rough idea for changes that are in 0.9 Most of the older ones have been backported, but I don't know if some of those merge commits are new for 0.9.
edit update list from git log on 1018-04-14 update list from git log on 1018-04-26
last larger backport was here https://github.com/statsmodels/statsmodels/pull/3444/commits https://github.com/statsmodels/statsmodels/commits/maintenance/0.8.x